Abnormal coal briquette processing method and device, medium, electronic equipment and program product

By generating a three-dimensional model of coal blocks and performing dimension comparison, the problem of insufficient accuracy of two-dimensional description in the prior art is solved, effective identification and early warning of abnormal coal blocks is achieved, and the safety and efficiency of coal transportation is ensured.

CN118982789APending Publication Date: 2024-11-19SHENHUA GUONENG ENERGY GRP +1
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
CN202411073471.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-06
Publication Date
2024-11-19

AI Technical Summary

Technical Problem

The existing coal block identification system is based on two-dimensional planar description, with low accuracy, which can easily cause false warnings and cannot effectively deal with safety hazards and transportation obstacles caused by abnormal coal blocks during transportation.

Method used

By obtaining multi-frame coal block images, a three-dimensional model of coal blocks is generated and size comparisons are made with the digital twin model of the target object, and early warning information is output to avoid transportation obstacles.

Benefits of technology

Improve the accuracy of coal block description, reduce false warnings, and ensure the safety and efficiency of the transportation process.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118982789A_ABST
    Figure CN118982789A_ABST
Patent Text Reader

Abstract

The invention relates to an abnormal coal briquette processing method and device, a medium, electronic equipment and a program product, and the method comprises the steps: obtaining a multi-frame coal briquette image which is an image for the same coal briquette collected when a scraper in a coal briquette transportation system transports coal flow; performing image analysis on the coal briquette image to obtain a turning surface of the coal briquette in the coal briquette image; determining the vertexes of the turning surfaces of the coal briquette images to obtain image feature points of the coal briquette images; determining three-dimensional coordinates of image feature points of the coal briquette images, and generating a three-dimensional model of the coal briquettes according to the three-dimensional coordinates of the image feature points of all the coal briquette images; performing size comparison according to the three-dimensional model of the coal briquette and the digital twin model of the target object to obtain a comparison result; under the condition that the comparison result represents that the target object can hinder the transportation of the coal briquettes, the early warning information is output, and the coal briquettes are described based on the three-dimensional dimensions, so that the description accuracy can be improved, the accuracy of the comparison result is further improved, and false early warning is avoided.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of coal mine technology, and in particular, to an abnormal coal block processing method, device, medium, electronic equipment and program product. Background Art

[0002] During coal mining and transportation, the coal mining machine will inevitably mistakenly send coal blocks that exceed the normal size to the conveyor belt. If these coal blocks are not cleaned up in time, they will accumulate with other normal coal blocks on the conveyor belt, wear the conveyor belt, and eventually cause damage to the conveyor belt, affecting normal transportation, and even bring safety hazards and cause danger. Therefore, manpower is needed to monitor and eliminate them. Therefore, it is necessary to provide a monitoring system. When the system identifies coal blocks that exceed the normal size, it will record and analyze them in the first time, and immediately issue an alarm. At the same time, it will work with the coal mining machine to stop the coal cutting action in time, and notify relevant personnel to go to the site to check and deal with it.

[0003] In the related technology, for example, the patent with application number CN202310062517.2 and name "Coal block identification and distance warning system for fully mechanized mining working face based on edge computing" provides a coal block identification and distance warning system for fully mechanized mining working face based on edge computing, including: the hardware system includes 3 groups of camera groups, edge processing nodes, video processing servers and UI (User Interface) servers; edge processing node identification process: identify coal blocks and coal mining machine drums through target detection models; while performing target detection model identification, identify Aruco codes to detect and locate the bracket numbers; match the bracket numbers and coal blocks, bracket numbers and coal mining machine drums at the same position on the video frame image to obtain the bracket numbers at the corresponding positions of the coal blocks and the bracket numbers at the corresponding positions of the coal mining machine drums; calculate the difference between the bracket numbers at the corresponding positions of the coal blocks and the bracket numbers at the corresponding positions of the coal mining machine drums as the distance between the coal blocks and the coal mining machine drums; when the number of brackets between the coal blocks and the coal mining machine drums is less than the set threshold, issue an early warning. However, the coal blocks identified by edge computing or video analysis are all two-dimensional planar descriptions, which are not very accurate and can easily cause false alarms. Summary of the invention

[0004] The purpose of the present disclosure is to provide a method, device, medium, electronic equipment and program product for processing abnormal coal blocks.

[0005] In order to achieve the above objectives, in a first aspect, the present disclosure provides a method for processing abnormal coal blocks, comprising: Acquire multiple frames of coal block images, wherein the multiple frames of coal block images are images of the same coal block acquired when a scraper conveyor in a coal block conveying system is conveying a coal flow; Performing image analysis on the coal block image to obtain a turning surface of the coal block in the coal block image; Determine the vertices of the turning surface of the coal block image to obtain image feature points of the coal block image; Determine the three-dimensional coordinates of the image feature points of the coal block image, and generate a three-dimensional model of the coal block according to the three-dimensional coordinates of all the image feature points of the coal block image; Performing a size comparison between the three-dimensional model of the coal block and the digital twin model of the target object to obtain a comparison result, wherein the comparison result is used to characterize whether the target object can hinder the transportation of the coal block; When the comparison result indicates that the target object can hinder the transportation of the coal block, a warning message is output.

[0006] Optionally, determining the three-dimensional coordinates of the image feature points of the coal block image includes: Establishing a three-dimensional coordinate reference system, and determining the initial three-dimensional coordinates of the image feature points of the coal block image in the three-dimensional coordinate reference system based on the three-dimensional coordinate reference system; The three-dimensional coordinates of the image feature points of the coal block image are determined according to the initial three-dimensional coordinates of the image feature points of the coal block image and the initial three-dimensional coordinates of the image feature points of the auxiliary coal block image corresponding to the coal block image.

[0007] Optionally, the auxiliary coal block image includes N frames, where N is greater than 0, and the auxiliary coal block image is a coal block image starting from the coal block image with a preset number of frames at each interval.

[0008] Optionally, the establishing of a three-dimensional coordinate reference system and determining, based on the three-dimensional coordinate reference system, initial three-dimensional coordinates of image feature points of the coal block image in the three-dimensional coordinate reference system comprises: Establishing a three-dimensional coordinate reference system based on the scraper; Grid lines are divided according to the three-dimensional coordinate reference system, and initial three-dimensional coordinates of image feature points of the coal block image in the three-dimensional coordinate reference system are determined according to the divided grid lines.

[0009] Optionally, performing image analysis on the coal block image to obtain a turning surface of the coal block in the coal block image includes: Performing image enhancement on the coal block image to obtain an enhanced image of the coal block image, wherein the image enhancement is used to enhance the light and dark contrast of the coal block image; The enhanced image is subjected to image analysis to obtain the turning surface of the coal block in the coal block image.

[0010] Optionally, performing size comparison between the three-dimensional model of the coal block and the digital twin model of the target object to obtain a comparison result includes: The volume represented by the three-dimensional model of the coal block is compared with the volume through which the digital twin model of the target object allows the coal block to pass, and a comparison result is obtained.

[0011] Optionally, the target object includes the coal transport chute of the scraper, the coal outlet of the scraper, and a coal block channel formed by the bottom of the coal mining machine and the scraper.

[0012] Optionally, generating the three-dimensional model of the coal block according to the three-dimensional coordinates of the image feature points of all the coal block images includes: According to the frame order of all the coal block images, the three-dimensional coordinates of the image feature points of all the coal block images are input into the three-dimensional engine in sequence, so that the three-dimensional engine generates triangular surfaces according to the three-dimensional coordinates of the image feature points of all the coal block images, and the three-dimensional model is formed by all the triangular surfaces.

[0013] Optionally, the method further includes: Using the physical engine of the digital twin system to perform collision simulation on the transportation of the coal block in the coal block transportation system, and generating a collision simulation animation; The collision simulation animation is displayed on a visual interface.

[0014] In a second aspect, the present disclosure provides an abnormal coal block processing device, comprising: A first acquisition module is configured to acquire multiple frames of coal block images, wherein the multiple frames of coal block images are images of the same coal block acquired when a scraper conveyor in a coal block transport system transports a coal flow; An analysis module is configured to perform image analysis on the coal block image to obtain a turning surface of the coal block in the coal block image; A first determination module is configured to determine the vertices of the turning surface of the coal block image to obtain image feature points of the coal block image; A second determination module is configured to determine the three-dimensional coordinates of the image feature points of the coal block image, and generate a three-dimensional model of the coal block according to the three-dimensional coordinates of the image feature points of all the coal block images; A comparison module is configured to perform size comparison between the three-dimensional model of the coal block and the digital twin model of the target object to obtain a comparison result, wherein the comparison result is used to characterize whether the target object can hinder the transportation of the coal block; The early warning module is configured to output early warning information when the comparison result indicates that the target object can hinder the transportation of the coal block.

[0015] In a third aspect, the present disclosure provides a non-temporary computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any one of the methods described in the first aspect.

[0016] In a fourth aspect, the present disclosure provides an electronic device, including: a memory having a computer program stored thereon; A processor is used to execute the computer program in the memory to implement the steps of any one of the methods in the first aspect.

[0017] In a fifth aspect, the present disclosure provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of any one of the methods in the first aspect.

[0018] Through the above technical scheme, a three-dimensional model of the coal block in the coal block image is generated based on the two-dimensional coal block image, and a size comparison is performed between the three-dimensional model of the coal block and the digital twin model of the target object to obtain a comparison result for characterizing whether the target object can hinder the transportation of the coal block. When the comparison result characterizes that the target object can hinder the transportation of the coal block, an early warning message is output to notify the staff to handle it so as to avoid congestion caused by large coal blocks during transportation. The description of the coal block based on the three-dimensional dimension can improve the accuracy of the description, thereby improving the accuracy of the comparison result and avoiding false warnings.

[0019] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present disclosure but do not constitute a limitation of the present disclosure. In the accompanying drawings: Figure 1 It is a schematic diagram showing an implementation environment according to an exemplary embodiment of the present disclosure.

[0021] Figure 2 The figure is a flow chart of a method for processing abnormal coal blocks according to an exemplary embodiment of the present disclosure.

[0022] Figure 3 The figure is a schematic diagram of an image of a turning surface of a coal block according to an exemplary embodiment of the present disclosure.

[0023] Figure 4 The figure is a schematic diagram showing the comparison of a coal block image before and after image enhancement according to an exemplary embodiment of the present disclosure.

[0024] Figure 5 It is a schematic diagram of a three-dimensional coordinate reference system according to an exemplary embodiment of the present disclosure.

[0025] Figure 6is a schematic diagram showing three frames of coal block images according to an exemplary embodiment of the present disclosure.

[0026] Figure 7 It is a schematic diagram of a Mesh model of a coal block according to an exemplary embodiment of the present disclosure.

[0027] Figure 8 The figure is a schematic diagram of an image showing a collision simulation animation according to an exemplary embodiment of the present disclosure.

[0028] Fig. 9 It is a block diagram of an abnormal coal block processing device according to an exemplary embodiment of the present disclosure.

[0029] Fig.10 It is a schematic structural diagram of an electronic device according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0030] The specific implementation of the present disclosure is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described herein is only used to illustrate and explain the present disclosure, and is not used to limit the present disclosure.

[0031] It should be noted that all actions of acquiring signals, information or data in the present disclosure are carried out in compliance with the relevant data protection laws and policies of the country where the device is located and with the authorization given by the owner of the corresponding device.

[0032] Figure 1 is a schematic diagram of an implementation environment according to an exemplary embodiment of the present disclosure. The abnormal coal block processing method depends on the implementation environment, see Figure 1 The implementation environment includes a camera group, an image processor, a three-dimensional engine system and a digital twin system. The camera group is used to collect coal block images, and the image processor is used to process the collected coal block images to obtain the turning surface of the coal block in the coal block image, determine the vertices of the turning surface of the coal block image, obtain the image feature points of the coal block image, and determine the three-dimensional coordinates of the image feature points of the coal block image; the determined three-dimensional coordinates are input into the three-dimensional engine system, the three-dimensional in the three-dimensional engine system is used to generate the corresponding three-dimensional model, and the three-dimensional model is input into the digital twin system to perform collision simulation, and obtain and display the collision simulation animation.

[0033] Among them, the digital twin system is a system that digitally simulates physical entities.

[0034] Through the above technical scheme, a three-dimensional model of the coal block in the coal block image is generated based on the two-dimensional coal block image, and a size comparison is performed between the three-dimensional model of the coal block and the digital twin model of the target object to obtain a comparison result for characterizing whether the target object can hinder the transportation of the coal block. When the comparison result characterizes that the target object can hinder the transportation of the coal block, an early warning message is output to notify the staff to handle it so as to avoid congestion caused by large coal blocks during transportation. The description of the coal block based on the three-dimensional dimension can improve the accuracy of the description, thereby improving the accuracy of the comparison result and avoiding false warnings.

[0035] Figure 2 is a flowchart of a method for processing abnormal coal blocks according to an exemplary embodiment of the present disclosure. The method can be applied to electronic devices, referring to Figure 2 , the method may include the following steps: Step 210, acquiring multiple frames of coal block images, wherein the multiple frames of coal block images are images of the same coal block acquired when a scraper conveyor in a coal block conveying system is conveying a coal flow; Step 220, performing image analysis on the coal block image to obtain a turning surface of the coal block in the coal block image; Step 230, determining the vertices of the turning surface of the coal block image to obtain image feature points of the coal block image; Step 240, determining the three-dimensional coordinates of the image feature points of the coal block image, and generating a three-dimensional model of the coal block according to the three-dimensional coordinates of the image feature points of all the coal block images; Step 250, performing a size comparison between the three-dimensional model of the coal block and the digital twin model of the target object to obtain a comparison result, wherein the comparison result is used to characterize whether the target object can hinder the transportation of the coal block; Step 260: when the comparison result indicates that the target object can hinder the transportation of coal blocks, output warning information.

[0036] Through the above technical scheme, a three-dimensional model of the coal block in the coal block image is generated based on the two-dimensional coal block image, and a size comparison is performed between the three-dimensional model of the coal block and the digital twin model of the target object to obtain a comparison result for characterizing whether the target object can hinder the transportation of the coal block. When the comparison result characterizes that the target object can hinder the transportation of the coal block, an early warning message is output to notify the staff to handle it so as to avoid congestion caused by large coal blocks during transportation. The description of the coal block based on the three-dimensional dimension can improve the accuracy of the description, thereby improving the accuracy of the comparison result and avoiding false warnings.

[0037] Among them, the coal block transportation system can include coal shearers and scrapers. Coal shearers and scrapers are two important mechanical equipment in coal mining. They work together in coal mining operations to complete the coal cutting, loading and transportation process. Specifically, the coal shearer is a device directly used for cutting coal seams underground. Its main function is to cut coal from the coal seam. The scraper is a continuous transportation equipment, which is mainly used to transport the coal cut by the coal shearer from the working face to the transfer machine or directly to the ground.

[0038] The number of coal block images may be set according to actual conditions, and this embodiment does not limit this.

[0039] The warning information may be output in the form of sound or text, etc. That is, this embodiment does not limit the output method of the warning information.

[0040] In some embodiments, the coal image may be provided by Figure 1 The cameras in the camera group shown in the figure can capture the data. The cameras can be installed on the cross arm of the top beam of the comprehensive mining working face support. The cross arm of the top beam of the comprehensive mining working face support is a mechanical component in the coal block transportation system.

[0041] Among them, the turning surface marks the transition of the surface of an object from a smooth continuous surface to another surface of different direction or curvature. Such a transition can be sharp edges, soft curves or complex geometric changes, adding a sense of layering and three-dimensionality to the object. The boundary between light and dark is the place where the gray side and the dark side of the object meet. It is usually a narrow dark strip from one side to the other, so it is called a line. Although it is called a "line", it is actually also a surface, a turning surface that connects the previous and the next. Figure 3 FIG. 1 is a schematic diagram of a turning surface of a coal block according to an exemplary embodiment of the present disclosure. Figure 3 , the image feature points of the coal block image include 01, 02, 03, 04, 05, 06, 07, 08, 09, 10, 11, 12. Taking 01 and 02 as an example, 01 and 02 constitute a turning surface.

[0042] In some embodiments, the above-mentioned step of performing image analysis on the coal block image to obtain the turning surface of the coal block in the coal block image can be implemented by the following methods: performing image enhancement on the coal block image to obtain an enhanced image of the coal block image, and image enhancement is used to enhance the light and dark contrast of the coal block image; performing image analysis on the enhanced image to obtain the turning surface of the coal block in the coal block image.

[0043] Figure 4 FIG. 1 is a schematic diagram showing a comparison of a coal block image before and after image enhancement according to an exemplary embodiment of the present disclosure. Figure 4 , Figure 4 The original image is a coal image without image enhancement. Figure 4 The processed image in the figure is a coal block image after image enhancement. It can be seen that the light and dark contrast of the processed image is more obvious than that of the original image.

[0044] By using the above method, the light-dark contrast of the coal block image is improved, thereby improving the accuracy of image analysis, and further improving the accuracy of the turning surface of the coal block in the determined coal block image.

[0045] In some embodiments, the above-mentioned step of determining the three-dimensional coordinates of the image feature points of the coal block image can be implemented in the following manner: establish a three-dimensional coordinate reference system, and determine the initial three-dimensional coordinates of the image feature points of the coal block image in the three-dimensional coordinate reference system based on the three-dimensional coordinate reference system; determine the three-dimensional coordinates of the image feature points of the coal block image based on the initial three-dimensional coordinates of the image feature points of the coal block image and the initial three-dimensional coordinates of the image feature points of the auxiliary coal block image corresponding to the coal block image.

[0046] Among them, a three-dimensional coordinate reference system can be established based on the scraper. Figure 5 , Figure 5 The three-dimensional coordinate reference system is established based on the scraper's scraper length of 0.75m, width of 0.8m, and height of 0.4m. Figure 5 In the figure, 01, 02, 03, 04, 05, 06, 07, 08, 09, 10, 11, and 12 displayed in red are image feature points respectively, 01 displayed in blue represents the coordinate origin of the three-dimensional coordinate reference system, the straight line where 01 displayed in blue and 04 displayed in blue are located can be the Z axis of the three-dimensional coordinate reference system, the straight line where 01 displayed in blue and 05 displayed in blue are located can be the X axis of the three-dimensional coordinate reference system, the straight line where 01 displayed in blue and 02 displayed in blue are located can be the Y axis of the three-dimensional coordinate reference system, and the coordinates of 01, 02, 03, 04, 05, 06, 07, and 08 displayed in blue are respectively: 01: (0,0,0); 02: (0,0.4,0); 03: (0, 0.4, 0.75); 04: (0,0,0.75); 05: (-0.8, 0, 0); 06: (-0.8, 0, 0.75); 07: (-0.8, 0.4, 0); 08:(-0.8,0.4,0.75).

[0047] After the three-dimensional coordinate reference system is established with the scraper of the scraper, the grid lines can be divided according to the established three-dimensional coordinate reference system, and the initial three-dimensional coordinates of the image feature points of the coal block image in the three-dimensional coordinate reference system can be determined according to the divided grid lines. It is worth noting that the grid lines are established to describe the scale intervals of the three-dimensional coordinate reference system on the X-axis, Y-axis and Z-axis, so that the values ​​of the image feature points on the X-axis, Y-axis and Z-axis in the three-dimensional coordinate reference system can be determined, and the corresponding initial three-dimensional coordinates can be determined according to the values ​​of the image feature points on the X-axis, Y-axis and Z-axis in the three-dimensional coordinate reference system.

[0048] The image feature points 01, 02, 03, 04, 05, 06, 07, 08, 09, 10, 11, 12 are displayed in red. Figure 5 The coordinates in the three-dimensional coordinate reference system shown are: 01: (-0.65, 0.35, 0.05); 02: (-0.55, 0.03, 0.02); 03: (-0.35, 0.3, 0.1); 04: (-0.35, 0.3, 0.1); 05: (-0.08, 0.15, 0.4); 06: (-0.1, 0.05, 0.1); 07: (-0.2, 0.28, 0.6); 08: (-0.45, 0.32, 0.7); 09: (-0.65, 0.34, 0.62); 10: (-0.85, 0.35, 0.38); 11: (-0.75, 0.38, 0.2); 12: Vector3(-0.82,0.02,0.22).

[0049] It is worth noting that during the transportation of coal blocks, the coal blocks are dynamic and may change their positions. In order to accurately obtain the three-dimensional coordinates of the image feature points of the coal blocks in the coal block image, multiple frames can be combined to optimize the calculated initial three-dimensional coordinates. As an example, the initial three-dimensional coordinates of the image feature points of the coal block image and the initial three-dimensional coordinates of the image feature points of the auxiliary coal block image corresponding to the coal block image can be weighted to obtain the three-dimensional coordinates of the image feature points of the coal block image.

[0050] The auxiliary coal block image may include N frames, where N is greater than 0, and the auxiliary coal block image is a coal block image starting from the coal block image and having a preset number of frames at intervals. N and the preset number of frames may be set according to actual conditions, and this embodiment does not limit this. It is worth noting that the position of the coal block in the coal block image and the auxiliary coal block image corresponding to the coal block image cannot be too far away from the established three-dimensional coordinate reference system.

[0051] For example, when the coal block image is the first frame coal block image, the auxiliary coal block images corresponding to the coal block image may be the fifth frame coal block image and the tenth frame coal block image. There are 4 frames of images between the first frame coal block image and the fifth frame coal block image, and between the fifth frame coal block image and the tenth frame coal block image, that is, the preset number of frames is 4, and N is 2.

[0052] by Figure 6 Taking the three frames of coal block images shown as an example, the fifth frame of coal block image and the tenth frame of coal block image serve as auxiliary coal block images of the first frame of coal block image, and the first frame of coal block image and the fifth frame of coal block image, as well as the fifth frame of coal block image and the tenth frame of coal block image are all separated by 4 frames of coal block images.

[0053] Reference Figure 6 , an exemplary explanation is given by taking the calculation of the three-dimensional coordinates of the image feature point 01 of the coal block in the first frame of the coal block image as an example.

[0054] The initial three-dimensional coordinates of the image feature point 01 in the first frame of the coal block image are (x 1 ,y 1 ,z 1 ), the initial three-dimensional coordinates of the image feature point 01 in the fifth frame of the coal block image are (x 2 ,y 2 ,z 2 ), the initial three-dimensional coordinates of the image feature point 01 in the tenth frame of the coal block image are (x 3 ,y 3 ,z 3 ).

[0055] According to the initial three-dimensional coordinates of the image feature point 01 in the first frame of the coal block image, the initial three-dimensional coordinates of the image feature point 01 in the fifth frame of the coal block image, and the initial three-dimensional coordinates of the image feature point 01 in the tenth frame of the coal block image, a weighted average is obtained to obtain the three-dimensional coordinates (x 4 ,y 4 ,z 4 ),Right now: x 4 =(x 1 +x 2 +x 3 ) / 3; y 4 =(y1 +y 2 +y 3 ) / 3; z 4 =(z 1 +z 2 +z 3 ) / 3.

[0056] It is worth noting that the three-dimensional coordinates of the image feature points of all coal block images can be determined according to the method of determining the three-dimensional coordinates of the image feature point 01 in the above example.

[0057] By means of the above method, the initial three-dimensional coordinates of the image feature points of the coal block image are optimized, thereby obtaining more accurate three-dimensional coordinates of the image feature points of the coal block image, thereby improving the accuracy of the three-dimensional model constructed based on the three-dimensional coordinates.

[0058] In some embodiments, the above-mentioned step of generating a three-dimensional model of a coal block based on the three-dimensional coordinates of the image feature points of all coal block images can be implemented in the following manner: according to the frame order of all coal block images, the three-dimensional coordinates of the image feature points of all coal block images are input into the three-dimensional engine in sequence, so that the three-dimensional engine generates triangular faces based on the three-dimensional coordinates of the image feature points of all coal block images, and the three-dimensional model is formed by all the triangular faces.

[0059] Among them, the three-dimensional model in the present disclosure may refer to a Mesh model. Usually, a 3D (three-dimensional) model is composed of polygons. A complex polygon is actually composed of multiple triangular faces. The surface of a 3D model is composed of multiple triangular faces connected to each other. The set of vertices and edges that constitute these triangular faces in three-dimensional space is the Mesh model.

[0060] Figure 7 It is a schematic diagram of a Mesh model of a coal block according to an exemplary embodiment of the present disclosure. Figure 7 The surface of the Mesh model is composed of multiple interconnected triangular faces, and each vertex of the triangular face is determined based on the three-dimensional coordinates of the image feature points of all coal block images.

[0061] The 3D engine can be Unity engine or Unreal engine. The 3D engine can be installed in Figure 1 The three-dimensional engine system shown.

[0062] In the above manner, a three-dimensional engine is used to generate a corresponding three-dimensional model according to the three-dimensional coordinates of the image feature points of multiple frames of coal block images.

[0063] In some embodiments, the above-mentioned step of performing size comparison between the three-dimensional model of the coal block and the digital twin model of the target object to obtain the comparison result can be implemented in the following way: comparing the volume represented by the three-dimensional model of the coal block with the volume through which the digital twin model of the target object allows the coal block to pass, to obtain the comparison result.

[0064] Among them, the target objects include the scraper's coal chute, the scraper's coal outlet, and the coal block channel formed by the bottom of the coal mining machine and the scraper.

[0065] Among them, the digital twin model is a virtual model based on physical entities. It integrates sensor data, design information, operation history and analysis algorithms to map and simulate the behavior, performance and status of equipment in real time in a virtual environment.

[0066] Among them, the comparison results include that the target object can hinder the transportation of coal blocks and that the target object cannot hinder the transportation of coal blocks. Correspondingly, the target object can hinder the transportation of coal blocks, which indicates that the transported coal blocks are larger, and the target object cannot hinder the transportation of coal blocks, which indicates that the transported coal blocks are smaller.

[0067] Among them, if the volume of coal blocks is larger than at least one of the capacity of the scraper's coal chute, the volume of the scraper's coal outlet allowed for the passage of coal blocks, and the volume of the coal block channel formed by the bottom of the coal mining machine and the scraper, it will cause congestion during the transportation of coal blocks, thereby affecting the transportation of coal blocks.

[0068] Therefore, through the above method, the volume of the coal block is compared with the volume that allows the coal block to communicate, so that large coal blocks that are likely to cause congestion can be identified and an early warning can be issued.

[0069] In some embodiments, the above method may also include: using the physical engine of the digital twin system to perform collision simulation on the transportation of coal blocks in the coal block transportation system to generate a collision simulation animation; and displaying the collision simulation animation on a visual interface.

[0070] It is worth mentioning that the crusher will further crush the coal blocks in the coal chute of the scraper. The crusher is located above the scraper. Therefore, larger coal blocks cannot enter the space formed by the crusher and the coal chute, and more and more larger coal blocks will inevitably cause accumulation. Figure 8 FIG. 1 is a schematic diagram of a collision simulation animation according to an exemplary embodiment of the present disclosure. Figure 8 In the example, the scraper is located below the crusher. The collision simulation is an animation of whether the coal blocks in the scraper's coal chute collide with the crusher during transportation. If a collision occurs, it will cause Figure 8As shown in the stacking effect, the coal blocks in the red box can never be broken, and more and more large coal blocks are bound to cause more serious stacking, thus affecting the mining efficiency of the coal blocks or causing accidents.

[0071] Therefore, through the above method, the collision simulation animation is displayed on the visual interface, so that the staff can visually observe whether the coal block will collide with the coal mining machine during transportation, and intervene in the impending accident in time.

[0072] Fig. 9 FIG. 1 is a block diagram of an abnormal coal block processing device according to an exemplary embodiment of the present disclosure. Fig. 9 The abnormal coal block processing device comprises: The first acquisition module 901 is configured to acquire multiple frames of coal block images, wherein the multiple frames of coal block images are images of the same coal block acquired when a scraper conveyor in a coal block transport system transports a coal flow; The analysis module 902 is configured to perform image analysis on the coal block image to obtain a turning surface of the coal block in the coal block image; The first determination module 903 is configured to determine the vertices of the turning surface of the coal block image to obtain image feature points of the coal block image; The second determination module 904 is configured to determine the three-dimensional coordinates of the image feature points of the coal block image, and generate a three-dimensional model of the coal block according to the three-dimensional coordinates of the image feature points of all the coal block images; A comparison module 905 is configured to perform a size comparison between the three-dimensional model of the coal block and the digital twin model of the target object to obtain a comparison result, wherein the comparison result is used to characterize whether the target object can hinder the transportation of the coal block; The warning module 906 is configured to output a warning message when the comparison result indicates that the target object can hinder the transportation of the coal block.

[0073] Optionally, the second determining module 904 includes: An establishing unit is configured to establish a three-dimensional coordinate reference system, and determine the initial three-dimensional coordinates of the image feature points of the coal block image in the three-dimensional coordinate reference system based on the three-dimensional coordinate reference system; The first determination unit is configured to determine the three-dimensional coordinates of the image feature points of the coal block image based on the initial three-dimensional coordinates of the image feature points of the coal block image and the initial three-dimensional coordinates of the image feature points of the auxiliary coal block image corresponding to the coal block image.

[0074] Optionally, the auxiliary coal block image includes N frames, where N is greater than 0, and the auxiliary coal block image is a coal block image starting from the coal block image with a preset number of frames at each interval.

[0075] Optionally, the establishing unit is specifically configured to: Establishing a three-dimensional coordinate reference system based on the scraper; Grid lines are divided according to the three-dimensional coordinate reference system, and initial three-dimensional coordinates of image feature points of the coal block image in the three-dimensional coordinate reference system are determined according to the divided grid lines.

[0076] Optionally, the analysis module 902 includes: An image enhancement unit is configured to perform image enhancement on the coal block image to obtain an enhanced image of the coal block image, wherein the image enhancement is used to enhance the light and dark contrast of the coal block image; The analysis unit is configured to perform image analysis on the enhanced image to obtain the turning surface of the coal block in the coal block image.

[0077] Optionally, the comparison module 905 is specifically configured to compare the volume represented by the three-dimensional model of the coal block with the volume through which the digital twin model of the target object allows the coal block to pass, to obtain a comparison result.

[0078] Optionally, the target object includes the coal chute of the scraper, the coal outlet of the scraper, and a coal block channel formed by the bottom of the coal mining machine and the scraper.

[0079] Optionally, the second determining module 904 further includes: The generating unit is configured to input the three-dimensional coordinates of the image feature points of all the coal block images into the three-dimensional engine in sequence according to the frame order of all the coal block images, so that the three-dimensional engine generates triangular faces according to the three-dimensional coordinates of the image feature points of all the coal block images, and the three-dimensional model is formed by all the triangular faces.

[0080] Optionally, the abnormal coal block processing device further includes: A generation module is configured to use a physical engine of a digital twin system to perform a collision simulation on the transportation of the coal block in the coal block transportation system to generate a collision simulation animation; The display module is configured to display the collision simulation animation on a visual interface.

[0081] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0082] Fig.10 FIG. 1 is a block diagram of an electronic device according to an exemplary embodiment. Fig.10As shown, the electronic device 1000 may include: a processor 1001 , a memory 1002 . The electronic device 1000 may also include one or more of a multimedia component 1003 , an input / output (I / O) interface 1004 , and a communication component 1005 .

[0083] The processor 1001 is used to control the overall operation of the electronic device 1000 to complete all or part of the steps in the abnormal coal block processing method described above. The memory 1002 is used to store various types of data to support the operation of the electronic device 1000, and these data may include, for example, instructions for any application or method used to operate on the electronic device 1000, and application-related data, such as contact data, sent and received messages, pictures, audio, video, etc. The memory 1002 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read-Only Memory, referred to as EPROM), programmable read-only memory (Programmable Read-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic memory, flash memory, magnetic disk or optical disk. The multimedia component 1003 may include a screen and an audio component. The screen may be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone, which is used to receive external audio signals. The received audio signal may be further stored in the memory 1002 or sent through the communication component 1005. The audio component also includes at least one speaker for outputting audio signals. The I / O interface 1004 provides an interface between the processor 1001 and other interface modules, and the above-mentioned other interface modules may be keyboards, mice, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 1005 is used for wired or wireless communication between the electronic device 1000 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IOT, eMTC, or other 5G, etc., or a combination of one or more of them, is not limited here. Therefore, the corresponding communication component 1005 may include: Wi-Fi module, Bluetooth module, NFC module, etc.

[0084] In an exemplary embodiment, the electronic device 1000 can be implemented by one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), controllers, microcontrollers, microprocessors or other electronic components to execute the above-mentioned abnormal coal block processing method.

[0085] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, and when the program instructions are executed by a processor, the steps of the abnormal coal block processing method described above are implemented. For example, the computer-readable storage medium may be the memory 1002 including the program instructions, and the program instructions may be executed by the processor 1001 of the electronic device 1000 to complete the abnormal coal block processing method described above.

[0086] In another exemplary embodiment, a computer program product is also provided. The computer program product includes a computer program that can be executed by a programmable device. The computer program has a code portion for executing the above-mentioned abnormal coal block processing method when executed by the programmable device.

[0087] The preferred embodiments of the present disclosure are described in detail above in conjunction with the accompanying drawings; however, the present disclosure is not limited to the specific details in the above embodiments. Within the technical concept of the present disclosure, a variety of simple modifications can be made to the technical solution of the present disclosure, and these simple modifications all fall within the protection scope of the present disclosure.

[0088] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present disclosure will not further describe various possible combinations.

[0089] In addition, various embodiments of the present disclosure may be arbitrarily combined, and as long as they do not violate the concept of the present disclosure, they should also be regarded as the contents disclosed by the present disclosure.

Claims

1. A method for processing abnormal coal blocks, It is characterized in that include: Acquire multiple frames of coal block images, wherein the multiple frames of coal block images are images of the same coal block acquired when a scraper conveyor in a coal block conveying system is conveying a coal flow; Performing image analysis on the coal block image to obtain a turning surface of the coal block in the coal block image; Determine the vertices of the turning surface of the coal block image to obtain image feature points of the coal block image; Determine the three-dimensional coordinates of the image feature points of the coal block image, and generate a three-dimensional model of the coal block according to the three-dimensional coordinates of all the image feature points of the coal block image; Performing a size comparison between the three-dimensional model of the coal block and the digital twin model of the target object to obtain a comparison result, wherein the comparison result is used to characterize whether the target object can hinder the transportation of the coal block; When the comparison result indicates that the target object can hinder the transportation of the coal block, a warning message is output.

2. The method according to claim 1, It is characterized in that The step of determining the three-dimensional coordinates of the image feature points of the coal block image comprises: Establishing a three-dimensional coordinate reference system, and determining the initial three-dimensional coordinates of the image feature points of the coal block image in the three-dimensional coordinate reference system based on the three-dimensional coordinate reference system; The three-dimensional coordinates of the image feature points of the coal block image are determined according to the initial three-dimensional coordinates of the image feature points of the coal block image and the initial three-dimensional coordinates of the image feature points of the auxiliary coal block image corresponding to the coal block image.

3. The method according to claim 2, It is characterized in that The auxiliary coal block image includes N frames, where N is greater than 0. The auxiliary coal block image is a coal block image with a preset number of frames at intervals starting from the coal block image.

4. The method according to claim 2, It is characterized in that The step of establishing a three-dimensional coordinate reference system and determining the initial three-dimensional coordinates of the image feature points of the coal block image in the three-dimensional coordinate reference system based on the three-dimensional coordinate reference system includes: Establishing a three-dimensional coordinate reference system based on the scraper; Grid lines are divided according to the three-dimensional coordinate reference system, and initial three-dimensional coordinates of image feature points of the coal block image in the three-dimensional coordinate reference system are determined according to the divided grid lines.

5. The method according to claim 1, It is characterized in that The performing image analysis on the coal block image to obtain the turning surface of the coal block in the coal block image includes: Performing image enhancement on the coal block image to obtain an enhanced image of the coal block image, wherein the image enhancement is used to enhance the light and dark contrast of the coal block image; The enhanced image is subjected to image analysis to obtain the turning surface of the coal block in the coal block image.

6. The method according to claim 1, It is characterized in that The size comparison is performed based on the three-dimensional model of the coal block and the digital twin model of the target object to obtain a comparison result, including: The volume represented by the three-dimensional model of the coal block is compared with the volume through which the digital twin model of the target object allows the coal block to pass, and a comparison result is obtained.

7. The method according to claim 1, It is characterized in that The target objects include the coal transport chute of the scraper, the coal outlet of the scraper, and a coal block channel formed by the bottom of the coal mining machine and the scraper.

8. The method according to claim 1, It is characterized in that Generating the three-dimensional model of the coal block according to the three-dimensional coordinates of the image feature points of all the coal block images includes: According to the frame order of all the coal block images, the three-dimensional coordinates of the image feature points of all the coal block images are input into the three-dimensional engine in sequence, so that the three-dimensional engine generates triangular surfaces according to the three-dimensional coordinates of the image feature points of all the coal block images, and the three-dimensional model is formed by all the triangular surfaces.

9. The method according to claim 1, It is characterized in that The method further comprises: Using the physical engine of the digital twin system to perform collision simulation on the transportation of the coal block in the coal block transportation system, and generating a collision simulation animation; The collision simulation animation is displayed on a visual interface.

10. An abnormal coal block processing device, It is characterized in that include: A first acquisition module is configured to acquire multiple frames of coal block images, wherein the multiple frames of coal block images are images of the same coal block acquired when a scraper conveyor in a coal block transport system transports a coal flow; An analysis module is configured to perform image analysis on the coal block image to obtain a turning surface of the coal block in the coal block image; A first determination module is configured to determine the vertices of the turning surface of the coal block image to obtain image feature points of the coal block image; A second determination module is configured to determine the three-dimensional coordinates of the image feature points of the coal block image, and generate a three-dimensional model of the coal block according to the three-dimensional coordinates of the image feature points of all the coal block images; A comparison module is configured to perform size comparison between the three-dimensional model of the coal block and the digital twin model of the target object to obtain a comparison result, wherein the comparison result is used to characterize whether the target object can hinder the transportation of the coal block; The early warning module is configured to output early warning information when the comparison result indicates that the target object can hinder the transportation of the coal block.

11. A non-transitory computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

12. An electronic device, It is characterized in that include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 9.

13. A computer program product comprising a computer program, It is characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Citation Information

Patent Citations

  • Method for monitoring efficient and safe operation of coal mining face ribbon conveyer

    CN110793587A

  • Coal briquette transportation monitoring method and device

    CN113942805A

  • Abnormal coal briquette identification method and device

    CN114973084A

  • Numerical control machine tool on-machine measurement implementation method based on digital twinborn model

    CN117452877A

  • Online monitoring method and system for coal conveying system

    CN117496430A