Image instance segmentation method and device for fully mechanized coal mining face

Abstract

The embodiment of the invention provides an image instance segmentation method and device for a fully mechanized coal mining face. The method comprises the following steps: acquiring target image data of the fully mechanized coal mining face; performing instance segmentation on the target image data through a pre-constructed initial segmentation model to generate an initial segmentation result; performing instance filtering according to the initial segmentation result through a preset region-based non-maximum suppression algorithm to generate an image instance segmentation result; through optimization design of characteristics of equipment such as the support side protection plates, accurate separation of the adjacent support side protection plates can be realized, other equipment such as a scraper conveyer, a coal mining machine roller, ground coal, ledges, a track and a cable trough can be identified at the same time, different examples of equipment of the same kind can be distinguished, and especially the overlapped adjacent support side protection plates can be distinguished. The embodiment-level accurate positioning capability is realized, the segmentation precision is improved, the calculation complexity is reduced, and the real-time monitoring requirement is met.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and particularly to an image instance segmentation method and device for fully-mechanized mining faces. Background Art

[0002] With the in-depth advancement of the intelligentization process of coal mines, higher requirements are put forward for the equipment recognition accuracy in the intelligent control and safety monitoring of fully-mechanized mining faces. Among them, as a key support equipment in the fully-mechanized mining face, the state monitoring of the support rib protection plate is directly related to roof safety and production efficiency. In practical applications, there are often overlaps and occlusions between adjacent support rib protection plates, making it difficult to accurately distinguish and locate each independent support rib protection plate instance. In addition, the working face also includes various equipment such as scraper conveyors, shearer drums, ground coal, trough sides, tracks, and cable troughs. The accurate recognition of these equipment is of great significance for realizing intelligent mining. In related technologies, equipment recognition is achieved through semantic segmentation methods or domain adaptation-based monitoring image segmentation methods, but different instances of the same type of equipment cannot be distinguished. When multiple support rib protection plates overlap, the boundaries of each support rib protection plate cannot be accurately located, the overlapping adjacent support rib protection plates cannot be distinguished, lacking instance-level precise positioning ability, and the segmentation accuracy is poor; moreover, the computational complexity is high, making it difficult to meet the requirements of real-time monitoring. Summary of the Invention

[0003] An object of the present invention is to provide an image instance segmentation method for fully-mechanized mining faces. By optimizing the features of equipment such as support rib protection plates, it can not only achieve the accurate separation of adjacent support rib protection plates, but also simultaneously identify other equipment such as scraper conveyors, shearer drums, ground coal, trough sides, tracks, and cable troughs, be able to distinguish different instances of the same type of equipment, especially be able to distinguish overlapping adjacent support rib protection plates, realize instance-level precise positioning ability, improve the segmentation accuracy, reduce the computational complexity, and meet the requirements of real-time monitoring. Another object of the present invention is to provide an image instance segmentation device for fully-mechanized mining faces. Another object of the present invention is to provide a computer-readable medium. Still another object of the present invention is to provide a computer device.

[0004] To achieve the above objects, on the one hand, the present invention discloses an image instance segmentation method for fully-mechanized mining faces, including:

[0005] Obtain target image data of the fully-mechanized mining face;

[0006] Perform instance segmentation on the target image data through a pre-constructed initial segmentation model to generate an initial segmentation result;

[0007] Perform instance filtering on the initial segmentation result through a preset region-based non-maximum suppression algorithm to generate an image instance segmentation result.

[0008] Preferably, it further includes:

[0009] Obtain an initial image dataset of the fully-mechanized coal mining face, where the initial image dataset includes multiple pieces of initial image data and unique color labels for each instance in each piece of initial image data;

[0010] Perform offline data augmentation on the initial image dataset to generate an offline image dataset;

[0011] Use the offline image dataset to train a model for the improved instance segmentation algorithm to construct an initial segmentation model. The improved instance segmentation algorithm includes a residual network and a feature pyramid network.

[0012] Preferably, obtaining the initial image dataset of the fully-mechanized coal mining face includes:

[0013] Collect the original image dataset of the fully-mechanized coal mining face, where the original image dataset includes the initial image data of the fully-mechanized coal mining face under different working conditions and different lighting conditions;

[0014] Use an interactive segmentation tool to label the unique color label for each instance in the initial image data;

[0015] Generate an initial image dataset based on the initial image data and the unique color label for each instance in the initial image data.

[0016] Preferably, performing offline data augmentation on the initial image dataset to generate an offline image dataset includes:

[0017] Perform geometric transformation on each piece of initial image data in the initial image dataset to generate enhanced image data. The geometric transformation includes affine transformation, shear transformation, uniaxial perspective transformation, and biaxial perspective transformation;

[0018] Convert the unique color label to a single-channel format label through the palette mode;

[0019] Generate an offline image dataset based on the enhanced image data and the single-channel format label.

[0020] Preferably, using the offline image dataset to train a model for the improved instance segmentation algorithm to construct an initial segmentation model includes:

[0021] Perform online data augmentation on the offline image dataset to generate a training image dataset;

[0022] Use a preset loss function and optimizer to train the model of the improved instance segmentation algorithm according to the training image dataset to generate an initial segmentation model.

[0023] Preferably, the initial segmentation result includes: a target mask, class information, and a confidence score;

[0024] Through a preset region-based non-maximum suppression algorithm, instance filtering is performed according to the initial segmentation result to generate an image instance segmentation result, including:

[0025] Calculate the intersection over union (IoU) for any two target masks to generate a mask IoU;

[0026] Through a preset attenuation function, an attenuation coefficient is generated according to a preset temperature parameter and the mask IoU;

[0027] Update the confidence score according to the attenuation coefficient to generate an updated confidence score;

[0028] Through a preset instance filtering threshold, instance filtering is performed on the initial segmentation result according to the updated confidence score to generate an image instance segmentation result.

[0029] The present invention also discloses an image instance segmentation device for a fully-mechanized coal mining face, including:

[0030] A target data acquisition unit for acquiring target image data of a fully-mechanized coal mining face;

[0031] An instance segmentation unit for performing instance segmentation on the target image data through a pre-constructed initial segmentation model to generate an initial segmentation result;

[0032] An instance filtering unit for performing instance filtering according to the initial segmentation result through a preset region-based non-maximum suppression algorithm to generate an image instance segmentation result.

[0033] Preferably, it further includes:

[0034] An initial data acquisition unit for acquiring an initial image data set of a fully-mechanized coal mining face, where the initial image data set includes multiple initial image data and a unique color label for each instance in each initial image data;

[0035] An offline data augmentation unit for performing offline data augmentation on the initial image data set to generate an offline image data set;

[0036] A model training unit for training a model of an improved instance segmentation algorithm through the offline image data set to construct an initial segmentation model, where the improved instance segmentation algorithm includes a residual network and a feature pyramid network.

[0037] Preferably, the initial data acquisition unit is specifically configured to collect the original image dataset of the fully-mechanized mining face, where the original image dataset includes the initial image data of the fully-mechanized mining face under different working conditions and different lighting conditions; label the unique color label of each instance in the initial image data through an interactive segmentation tool; generate the initial image dataset according to the initial image data and the unique color label of each instance in the initial image data.

[0038] Preferably, the offline data augmentation unit is specifically configured to perform geometric transformations on each piece of initial image data in the initial image dataset to generate augmented image data, where the geometric transformations include affine transformation, shear transformation, uniaxial perspective transformation, and biaxial perspective transformation; convert the unique color label into a single-channel format label through the palette mode; generate the offline image dataset according to the augmented image data and the single-channel format label.

[0039] Preferably, the model training unit is specifically configured to perform online data augmentation on the offline image dataset to generate the training image dataset; perform model training on the improved instance segmentation algorithm according to the training image dataset through a preset loss function and optimizer to generate the initial segmentation model.

[0040] Preferably, the initial segmentation result includes: a target mask, class information, and a confidence score;

[0041] The instance filtering unit is specifically configured to calculate the intersection over union of any two target masks to generate the mask intersection over union; generate an attenuation coefficient according to a preset temperature parameter and the mask intersection over union through a preset attenuation function; update the confidence score according to the attenuation coefficient to generate the updated confidence score; perform instance filtering on the initial segmentation result according to the updated confidence score through a preset instance filtering threshold to generate the image instance segmentation result.

[0042] The present invention also discloses a computer-readable medium, on which a computer program is stored, and when the program is executed by a processor, the above-mentioned method is implemented.

[0043] The present invention also discloses a computer device, including a memory and a processor, where the memory is used to store information including program instructions, and the processor is used to control the execution of the program instructions, and when the processor executes the program, the above-mentioned method is implemented.

[0044] The present invention also discloses a computer program product, including computer programs / instructions, and when the computer programs / instructions are executed by a processor, the above-mentioned method is implemented.

[0045] The present invention obtains the target image data of the fully mechanized mining face; performs instance segmentation on the target image data through a pre-constructed initial segmentation model to generate an initial segmentation result; performs instance filtering according to the initial segmentation result through a preset region-based non-maximum suppression algorithm to generate an image instance segmentation result. By optimizing the features of equipment such as the support rib protection plate, it can not only achieve the precise separation of adjacent support rib protection plates, but also simultaneously identify other equipment such as the scraper conveyor, shearer drum, ground coal, trough side, track, cable trough, etc., and can distinguish different instances of the same type of equipment, especially can distinguish overlapping adjacent support rib protection plates, realize the precise positioning ability at the instance level, improve the segmentation accuracy, reduce the computational complexity, and meet the real-time monitoring requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0047] Figure 1 It is a flowchart of an image instance segmentation method for a fully mechanized mining face provided by an embodiment of the present invention;

[0048] Figure 2 It is a flowchart of another image instance segmentation method for a fully mechanized mining face provided by an embodiment of the present invention;

[0049] Figure 3 It is a flowchart of a method for obtaining an initial image data set provided by an embodiment of the present invention;

[0050] Figure 4 It is a flowchart of an offline data augmentation provided by an embodiment of the present invention;

[0051] Figure 5 It is a flowchart of a method for training an initial segmentation model provided by an embodiment of the present invention;

[0052] Figure 6 It is a structural schematic diagram of an image instance segmentation device for a fully mechanized mining face provided by an embodiment of the present invention;

[0053] Figure 7 It is a structural schematic diagram of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0054] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0055] It should be noted that an image instance segmentation method and device for a fully mechanized mining face disclosed in this application can be used in the field of artificial intelligence technology, and can also be used in any field other than the field of artificial intelligence technology. The application field of the image instance segmentation method and device for a fully mechanized mining face disclosed in this application is not limited.

[0056] To facilitate understanding of the technical solutions provided in this application, the relevant content of the technical solutions in this application will be described first. The core technical problem to be solved by the present invention is the instance segmentation problem of fully mechanized mining face equipment, especially the precise segmentation problem of adjacent support rib plates. In the underground fully mechanized mining face environment, support rib plates, as important support equipment, often appear adjacent and overlapping, and the existing technology cannot effectively separate and identify overlapping support rib plates. At the same time, the working face environment is complex, with various interference factors such as insufficient light, dust interference, and equipment fouling, which pose great challenges to equipment identification. In addition, to meet the real-time monitoring requirements, it is also necessary to ensure that the algorithm has a high processing speed and low resource occupancy while ensuring the recognition accuracy. The innovation of the present invention also lies in the proposed region-based non-maximum suppression (NMS) algorithm. The traditional box-based NMS algorithm only considers the overlap of bounding boxes. For equipment with irregular shapes such as scraper conveyors, cable troughs, and support rib plates, there are often cases of misdeletion caused by the overlap of bounding boxes while the actual regions do not overlap. The region-based NMS algorithm can calculate the actual overlap region between masks and perform instance filtering in combination with a decay strategy with memory, successfully avoiding this misdeletion phenomenon. In addition, by introducing an exponential decay function to penalize the overlap region, the algorithm can more flexibly handle different degrees of overlap, thus achieving precise segmentation of adjacent equipment.

[0057] One of the key technical points of the present invention is the instance segmentation method adopted for the difficult problem of the segmentation of the support rib protection plates in the fully mechanized coal mining face. In the environment of the fully mechanized coal mining face, the support rib protection plate is the most critical support equipment, and its identification and monitoring are crucial for safe production. The identification of the support rib protection plate faces three main challenges: First, the support rib protection plates often overlap with each other; Second, the shapes of the support rib protection plates are irregular; Third, the underground environment is complex. The present invention deeply analyzes the technical characteristics of the SOLOV2 model and finds that its idea of "decoupled detection" is particularly suitable for dealing with the problem of support rib protection plate segmentation. SOLOV2 does not rely on proposal boxes but directly learns instance masks on the feature map, and this design is naturally suitable for dealing with objects with irregular shapes. At the same time, its lightweight dynamic convolution head design ensures a relatively fast inference speed. These characteristics enable SOLOV2 to have unique advantages in the task of support rib protection plate segmentation. Through instance-level modeling and pixel-level mask generation, good segmentation effects can be maintained even when the support rib protection plates are densely arranged. The experimental results show that this method based on instance segmentation enables the recognition accuracy of the support rib protection plate to reach 92.3%.

[0058] The present invention optimizes the design according to the characteristics of equipment such as support rib protection plates, and can not only achieve the precise separation of adjacent support rib protection plates, but also simultaneously identify other equipment such as scraper conveyors, shearer drums, ground coal, trough sides, tracks, cable troughs, etc. The present invention realizes a processing performance of 2.332G video memory occupancy and 33.8ms / frame on the graphics processing unit (GPU) platform. The recognition accuracy of the support rib protection plate reaches 92.3%, and the average recognition accuracy of other equipment reaches 89.7%. It shows excellent recognition performance and environmental adaptability in the complex underground environment, providing reliable technical support for the equipment monitoring of the intelligent fully mechanized coal mining face.

[0059] Taking the image instance segmentation device for the fully mechanized coal mining face as the execution subject as an example below, the implementation process of the image instance segmentation method for the fully mechanized coal mining face provided by the embodiments of the present invention is described. It can be understood that the execution subject of the image instance segmentation method for the fully mechanized coal mining face provided by the embodiments of the present invention includes but is not limited to the image instance segmentation device for the fully mechanized coal mining face.

[0060] Figure 1 It is a flowchart of an image instance segmentation method for a fully mechanized coal mining face provided by an embodiment of the present invention. As Figure 1 shown, the method includes:

[0061] Step 101, obtain the target image data of the fully mechanized coal mining face.

[0062] In the embodiments of the present invention, the target image data is the image data of the fully-mechanized mining face collected by an image acquisition device, and the target image data covers at least one of the key devices such as the support rib protection plate, scraper conveyor, shearer drum, ground coal, trough side, track, cable trough, etc. of the working face.

[0063] Step 102: Perform instance segmentation on the target image data through a pre-constructed initial segmentation model to generate an initial segmentation result.

[0064] In the embodiments of the present invention, the initial segmentation model is obtained by pre-training an improved instance segmentation algorithm, and the improved instance segmentation algorithm includes a residual network and a feature pyramid network. Specifically, the initial segmentation model selects SOLOV2 as the basis, uses a residual network (ResNet-50) as the backbone network, and cooperates with a feature pyramid network (FPN) to achieve multi-scale feature extraction.

[0065] Step 103: Perform instance filtering on the initial segmentation result through a preset region-based NMS algorithm to generate an image instance segmentation result.

[0066] In the embodiments of the present invention, the core idea of the region-based NMS algorithm is to directly calculate the actual overlapping area between masks, so as to accurately and efficiently process irregular shapes and targets; combined with a decay strategy with memory for instance filtering, the problem of misdeletion is overcome, and the segmentation accuracy is improved.

[0067] In the embodiments of the present invention, the region-based NMS algorithm has a low computational complexity and can meet the real-time requirement.

[0068] In the technical solution provided by the embodiments of the present invention, the target image data of the fully-mechanized mining face is obtained; instance segmentation is performed on the target image data through a pre-constructed initial segmentation model to generate an initial segmentation result; instance filtering is performed on the initial segmentation result through a preset region-based non-maximum suppression algorithm to generate an image instance segmentation result. By optimizing the features of devices such as the support rib protection plate, not only can the accurate separation of adjacent support rib protection plates be achieved, but also other devices such as the scraper conveyor, shearer drum, ground coal, trough side, track, cable trough, etc. can be identified simultaneously, and different instances of the same type of device can be distinguished, especially the overlapping adjacent support rib protection plates can be distinguished, realizing the precise positioning ability at the instance level, improving the segmentation accuracy, reducing the computational complexity, and meeting the real-time monitoring requirements.

[0069] Figure 2 It is a flowchart of another image instance segmentation method for a fully-mechanized mining face provided by the embodiments of the present invention. As Figure 2 shown, this method includes:

[0070] Step 201: Obtain the initial image dataset of the fully-mechanized mining face.

[0071] In the embodiment of the present invention, the initial image dataset includes multiple pieces of initial image data and the unique color (RGB) label of each instance in each piece of initial image data.

[0072] Figure 3 The flowchart for obtaining the initial image dataset provided by the embodiment of the present invention is as Figure 3 shown. Specifically, step 201 includes:

[0073] Step 2011: Collect the original image dataset of the fully-mechanized mining face.

[0074] In the embodiment of the present invention, the original image dataset includes the initial image data of the fully-mechanized mining face under different working conditions and different lighting conditions.

[0075] Specifically, use an industrial camera to collect the initial image data of the fully-mechanized mining face under different working conditions and different lighting conditions. The collected initial image data needs to cover various situations such as normal equipment operation and fault states, and particularly pay attention to the image collection of the overlapping area of the support rib protection plates.

[0076] Step 2012: Through an interactive segmentation tool, label the unique RGB label of each instance in the initial image data.

[0077] Specifically, use an interactive segmentation tool (EISeg) to accurately label each instance in the collected initial image data, separately identify each support rib protection plate instance, ensure that each support rib protection plate instance is given a unique identifier, and at the same time complete the pixel-level labeling of key equipment such as the scraper conveyor, shearer drum, ground coal, trough side, track, and cable trough.

[0078] As another alternative, in addition to using the EISeg interactive segmentation tool for labeling, other semi-automatic labeling tools can also be used for data labeling, such as LabelMe or Supervisely.

[0079] Step 2013: Generate the initial image dataset according to the initial image data and the unique RGB label of each instance in the initial image data.

[0080] Specifically, combine multiple pieces of initial image data and the unique RGB label of each instance corresponding to each piece of initial image data one by one to generate the initial image dataset.

[0081] Step 202: Perform offline data augmentation on the initial image dataset to generate an offline image dataset.

[0082] Figure 4A flowchart of offline data augmentation provided by an embodiment of the present invention is as follows Figure 4 As shown, step 202 specifically includes

[0083] Step 2021: Perform geometric transformation on each piece of initial image data in the initial image dataset to generate enhanced image data

[0084] The embodiment of the present invention designs offline data augmentation and online data augmentation. Offline data augmentation is applied before model training, and online data augmentation is applied during model training

[0085] In the embodiment of the present invention, the geometric transformation includes but is not limited to affine transformation, shear transformation, uniaxial perspective transformation, and biaxial perspective transformation. Among them, the affine transformation generates a new image through a combined transformation of a rotation angle of ±10 degrees and a scaling ratio of 0.8; the shear transformation generates a new image by adding a shear angle of ±5 degrees on the basis of the affine transformation; the uniaxial perspective transformation generates a new image by performing a 3D rotation of the image around the X-axis; the biaxial perspective transformation generates a new image by combining the 3D rotations of the X-axis and the Y-axis

[0086] As another alternative, the data augmentation method can also adopt other strategies, such as using a generative adversarial network (GAN) to generate more diverse training data, or adopting a physical model-based rendering method to generate images under different illuminations and viewpoints

[0087] In the embodiment of the present invention, the above geometric transformation processing is performed on each piece of initial image data and the transformed images are saved, thus expanding the dataset. Taking the above 4 geometric transformations as an example, after the initial image data undergoes the above 4 geometric transformations, the dataset is expanded to 5 times the original (1 original image + 4 transformed images), significantly improving the model's adaptability to different viewpoints

[0088] Step 2022: Convert the unique RGB label into a single-channel format label through the palette mode

[0089] Specifically, the palette mode conversion converts the unique RGB label into a single-channel format, improving the data processing efficiency

[0090] Step 2023: Generate an offline image dataset according to the enhanced image data and the single-channel format label

[0091] Specifically, multiple pieces of enhanced image data and the single-channel format labels of each instance corresponding to each piece of enhanced image data are combined one by one to generate an offline image dataset

[0092] It should be noted that offline data augmentation performs the same operations on image data and labels, while online data augmentation only operates on image data and does not perform operations on labels

[0093] By combining the use of affine transformation, shear transformation, and perspective transformation, the present invention amplifies the data by 5 times the original, significantly enhancing the model's adaptability to different perspectives and deformations; at the same time, online data augmentation is adopted to simulate the complex underground environment, improving the model's environmental adaptability.

[0094] Step 203: Through the offline image dataset, train the model for the improved instance segmentation (SOLOV2) algorithm to construct an initial segmentation model.

[0095] In the embodiments of the present invention, the improved instance segmentation algorithm includes a residual network (ResNet-50) and a feature pyramid network. Specifically, the initial segmentation model selects SOLOV2 as the basis, uses the residual network (ResNet-50) as the backbone network, and cooperates with the feature pyramid network (FPN) to achieve multi-scale feature extraction.

[0096] In the embodiments of the present invention, SOLOV2 is selected as the basis, which makes good use of the advantages of SOLOV2 with small computational complexity and fast speed, facilitating real-time data monitoring, and can be organically combined with region-based NMS to accurately and efficiently identify the problem of overlapping support rib protection plates.

[0097] Figure 5 It is a flowchart for training an initial segmentation model provided by the embodiments of the present invention. As Figure 5 shown, step 203 specifically includes:

[0098] Step 2031: Perform online data augmentation on the offline image dataset to generate a training image dataset.

[0099] In the embodiments of the present invention, online data augmentation is applied during the model training process. Online data augmentation only operates on the image data and does not perform operations on the labels.

[0100] Specifically, online data augmentation dynamically realizes operations such as random horizontal flipping of the image, random rotation of ±5 degrees, brightness adjustment range of ±20%, contrast adjustment range of ±10%, randomly adding Gaussian noise (mean 0, standard deviation 0.01) during the training process, and randomly fogging and overexposing the image, etc., to improve the model's recognition ability for situations such as insufficient light, dust interference, and equipment fouling.

[0101] Step 2032: Through a preset loss function and optimizer, train the improved instance segmentation algorithm according to the training image dataset to generate an initial segmentation model.

[0102] In the embodiments of the present invention, the training image dataset includes the image data after online data augmentation and the single-channel format labels corresponding to each instance. Among them, the resolution of the image data after online data augmentation is set to 800×800.

[0103] Specifically, the training image dataset is input into the improved SOLOV2 algorithm for model training. The original loss function design of SOLOV2 is adopted, including segmentation loss and classification loss. During the training process, a configuration with a batch size of 8 and a base learning rate of 0.01 is adopted, and the stochastic gradient descent optimizer is used for training optimization until the model accuracy meets the preset accuracy requirements, completing the construction of the initial segmentation model.

[0104] Furthermore, for the actual application in the industrial field, the present invention converts the trained PyTorch model into the ONNX format, which can improve the generality of the deployment environment and reduce the maintenance cost. During the conversion process, the input size is set to 800×800×3, and the output includes the target mask, class information, and confidence score.

[0105] The present invention realizes a relatively fast processing speed while ensuring the recognition accuracy through the ONNX format conversion and warm-up mechanism, meeting the real-time application requirements of the industrial field.

[0106] Step 204: Obtain the target image data of the fully-mechanized mining face.

[0107] In the embodiments of the present invention, the target image data is the image data of the fully-mechanized mining face collected by the image acquisition device, and the target image data covers at least one of the key devices such as the support rib protection plate, scraper conveyor, shearer drum, ground coal, trough side, track, and cable trough of the working face. Among them, the image acquisition device can be an industrial camera.

[0108] Step 205: Perform instance segmentation on the target image data through the pre-constructed initial segmentation model to generate an initial segmentation result.

[0109] Specifically, the target image data is input into the initial segmentation model for instance segmentation, and the initial segmentation result is output. The initial segmentation result includes: target mask, class information, and confidence score.

[0110] Step 206: Calculate the intersection over union (IoU) for any two target masks to generate the mask IoU.

[0111] Specifically, for any two target masks mask_i and mask_j, calculate their intersection over union IoU:

[0112] IoU(mask_i,mask_j)=|mask_i∩mask_j| / |mask_i∪mask_j|

[0113] Among them, |mask_i ∩ mask_j| represents the area of the intersection region of the target masks i and j, |mask_i ∪ mask_j| represents the area of the union region of the target masks i and j, and IoU(mask_i, mask_j) represents the intersection over union of the target masks i and j.

[0114] Step 207: Generate a decay coefficient according to a preset temperature parameter and the intersection over union through a preset decay function.

[0115] Specifically, calculate the preset temperature parameter and the intersection over union through decay(IoU) = exp(-(IoU^2) / σ) to generate a decay coefficient. Where decay(IoU) is the decay coefficient, IoU is the intersection over union, and σ is the temperature parameter.

[0116] It should be noted that the temperature parameter is adjustable and used to control the decay rate. The specific value of the temperature parameter in the embodiments of the present invention is not limited.

[0117] Step 208: Update the confidence score according to the decay coefficient to generate an updated confidence score.

[0118] In the embodiments of the present invention, the confidence score of each segmentation instance is updated:

[0119] score_new = score_old × min(decay(IoU_1), decay(IoU_2),..., decay(IoU_n))

[0120] Where score_new is the updated confidence score, score_old is the original confidence score, and decay(IoU_k) is the decay coefficient corresponding to the k-th overlapping instance.

[0121] Step 209: Filter the initial segmentation result according to the updated confidence score through a preset instance filtering threshold to generate an image instance segmentation result.

[0122] In the embodiments of the present invention, the instance filtering threshold can be set according to actual needs, and the embodiments of the present invention do not limit this. The filtering condition is: keep = score_new > threshold, where keep is the retained instance, score_new is the updated confidence score, and threshold is the instance filtering threshold.

[0123] Specifically, if the updated confidence score is greater than the instance filtering threshold, the instance corresponding to the updated confidence score is retained; if the updated confidence score is less than or equal to the instance filtering threshold, the instance corresponding to the updated confidence score is filtered out.

[0124] In the embodiments of the present invention, the final image instance segmentation result includes the target mask, class information, and updated confidence score of the retained instances.

[0125] Through region-based NMS, the present invention can directly calculate the mask overlap area, avoid the error caused by the bounding box, and improve the accuracy; introduce an exponential decay strategy to make the penalty for the overlapping area smoother; retain the instance shape information and improve the processing ability for targets with irregular shapes.

[0126] The following uses an embodiment applied in an actual scenario to specifically illustrate the execution process of the instance segmentation method:

[0127] Field application tests were carried out in a fully mechanized coal mining face in a certain mining area. The test environment was a fully mechanized coal mining face with a coal seam thickness of 3.5 meters in the working face. An industrial camera was used to collect data during the data collection process. The camera was installed on the shearer and the collection height was 1.2 meters from the ground. Images were collected at different times and under different working conditions, including various scenarios such as normal working conditions, equipment maintenance conditions, and emergency conditions. A total of 1897 initial image data were collected (367 of which were pure backgrounds). The collected images covered seven types of key equipment in the working face, including the support rib protection plate, scraper conveyor, shearer drum, ground coal, trough side, track, and cable trough. The image resolution was 2048×1536, and the data balance of different lighting conditions and different equipment states (clean state, dust-covered state) was ensured during the collection process.

[0128] After the acquisition is completed, use the EISeg interactive segmentation tool for accurate annotation. During the annotation process, special attention is paid to the overlapping areas of adjacent support rib protectors to ensure that each support rib protector instance is assigned a unique identifier. At the same time, pixel-level annotation is performed on the other six types of equipment. The initial image dataset is divided into a training set of 1500 images and a validation set of 397 images according to a ratio of 8:2. To enhance data diversity, offline data augmentation is performed on the training set. Affine transformation, shear transformation, perspective transformation around the X-axis, and biaxial perspective transformation are used to amplify the data by 5 times. Palette mode conversion converts the RGB-format annotation to a single-channel format. A color mapping relationship is established: (0,0,0) corresponds to the background, (85,255,0) corresponds to the support rib protector, (53,119,181) corresponds to the scraper conveyor, (255,0,255) corresponds to the track, (245,128,6) corresponds to the cable trough, (255,255,127) corresponds to the trough side, (255,0,0) corresponds to the ground coal, and (255,255,255) corresponds to the shearer drum. At the same time, online data augmentation is implemented during the training process, including random horizontal flipping, random rotation of ±5 degrees, brightness adjustment range of ±20%, contrast adjustment range of ±10%, randomly adding Gaussian noise (mean 0, standard deviation 0.01), and randomly simulating dust and overexposure effects.

[0129] In the model deployment environment, a server with a GPU with a video memory of not less than 4GB and a CPU configuration meeting the performance requirements is adopted, and the operating system is Ubuntu 20.04. The model is converted to the ONNX format and warmed up (perform 10 empty inferences) during inference. The input images are uniformly scaled to a resolution of 800×800 for processing. In the post-processing stage, the following parameter configurations are adopted for the region-based NMS algorithm: detection score threshold 0.3, mask threshold 0.5, and update threshold 2.0. This set of parameters has been verified through multiple experiments and can effectively balance accuracy and recall, and is particularly suitable for dealing with the overlapping recognition problems of equipment such as support rib protectors.

[0130] The test results show that the system exhibits excellent performance in the actual working face environment: the average image processing time per frame is 33.8 milliseconds, meeting the requirements of real-time monitoring; the video memory occupancy of the model is only 2.332GB, facilitating deployment in an environment with limited computing resources; the instance segmentation accuracy of the support rib protection plate reaches 92.3%, and different support rib protection plate instances can be accurately distinguished even in overlapping areas; the average recognition accuracy of the other six types of equipment reaches 89.7%. Especially when dealing with the problem of overlapping support rib protection plates, compared with the traditional box-based NMS, the false deletion rate of the region-based NMS algorithm is reduced by 15.6%, and the boundary positioning accuracy is improved by 12.8%. When tested under different lighting conditions, the recognition accuracies of the system under normal lighting, emergency lighting, and local lighting conditions are 93.2%, 88.5%, and 86.7% respectively. Under the working condition of dust interference, the recognition accuracy can still remain above 85%, demonstrating good environmental adaptability. The present invention breaks through the limitations of traditional methods and is of great significance for improving the intelligent and automated level of fully mechanized coal mining faces.

[0131] In terms of real-time performance, the present invention achieves excellent processing efficiency while ensuring high precision, greatly reducing the hardware cost. These performance indicators enable the system to operate stably under limited computing resource conditions, greatly enhancing the practicality and promotion value of the system.

[0132] In terms of environmental adaptability, the present invention demonstrates strong robustness. Through data augmentation and model optimization of the system, the system maintains a high recognition accuracy under different lighting conditions, fully meeting the application requirements of the complex underground environment.

[0133] In terms of intelligent production, the present invention provides key technical support for the intelligent upgrading of fully mechanized coal mining faces. The system can simultaneously identify and monitor seven types of key equipment such as support rib protection plates, scraper conveyors, and shearer drums, and the average recognition accuracy reaches 89.7%. This comprehensive equipment monitoring ability lays a solid foundation for realizing intelligent control and safety monitoring of the working face.

[0134] In terms of work safety, the present invention significantly improves the reliability and timeliness of equipment monitoring. The system can monitor the status of each support rib protection plate in real time, promptly detect abnormal situations, and effectively prevent equipment failures and safety accidents. This feature is of great value for improving the work safety level of mines.

[0135] The present invention not only solves the technical problems of equipment recognition in fully mechanized coal mining faces, but also has strong versatility. Its core algorithm and implementation method can be extended and applied to equipment recognition and monitoring in other industrial scenarios, with broad application prospects.

[0136] It should be noted that in the technical solution of this application, the acquisition, storage, use, processing, etc. of data all comply with the relevant regulations of laws and regulations. The user information in the embodiments of this application is obtained through legal and compliant channels, and the acquisition, storage, use, processing, etc. of user information have obtained the authorization and consent of the customers.

[0137] It should be noted that the information collected in this application is information and data authorized by the user or fully authorized by all parties. Moreover, for the processing of relevant data, such as collection, storage, use, processing, transmission, provision, disclosure, and application, etc., all comply with the relevant laws, regulations, and standards of relevant countries and regions. Necessary confidentiality measures have been taken, which do not violate public order and good customs, and corresponding operation entrances are provided for users to choose to authorize or refuse.

[0138] It should be noted that the technical solution provided by this application provides corresponding operation entrances for users to choose to agree or refuse the automated decision-making results; if the user chooses to refuse, the expert decision-making process will be entered.

[0139] In the technical solution of the image instance segmentation method for fully mechanized coal mining face provided by the embodiments of the present invention, target image data of the fully mechanized coal mining face is acquired; through a pre-constructed initial segmentation model, instance segmentation is performed on the target image data to generate an initial segmentation result; through a preset region-based non-maximum suppression algorithm, instance filtering is performed according to the initial segmentation result to generate an image instance segmentation result. By optimizing the design of the features of equipment such as the support rib protection plate, not only can the accurate separation of adjacent support rib protection plates be achieved, but also other equipment such as the scraper conveyor, shearer drum, ground coal, trough side, track, cable trough, etc. can be identified simultaneously. Different instances of the same type of equipment can be distinguished, especially the overlapping adjacent support rib protection plates can be distinguished, realizing the precise positioning ability at the instance level, improving the segmentation accuracy, reducing the computational complexity, and meeting the real-time monitoring requirements.

[0140] Figure 6 This is a structural schematic diagram of an image instance segmentation device for a fully mechanized coal mining face provided by the embodiments of the present invention. This device is used to execute the above-mentioned image instance segmentation method for a fully mechanized coal mining face, as Figure 6 shown, this device includes: a target data acquisition unit 11, an instance segmentation unit 12, and an instance filtering unit 13.

[0141] The target data acquisition unit 11 is used to acquire the target image data of the fully mechanized coal mining face.

[0142] The instance segmentation unit 12 is used to perform instance segmentation on the target image data through a pre-constructed initial segmentation model to generate an initial segmentation result.

[0143] The instance filtering unit 13 is used to perform instance filtering on the basis of the initial segmentation result through a preset region-based non-maximum suppression algorithm, and generate an image instance segmentation result.

[0144] In an embodiment of the present invention, it further includes: an initial data acquisition unit 14, an offline data enhancement unit 15, and a model training unit 16.

[0145] The initial data acquisition unit 14 is used to acquire an initial image dataset of the fully-mechanized coal mining face. The initial image dataset includes multiple initial image data and a unique color label for each instance in each initial image data.

[0146] The offline data enhancement unit 15 is used to perform offline data enhancement on the initial image dataset to generate an offline image dataset.

[0147] The model training unit 16 is used to perform model training on the improved instance segmentation algorithm through the offline image dataset to construct an initial segmentation model. The improved instance segmentation algorithm includes a residual network and a feature pyramid network.

[0148] In an embodiment of the present invention, the initial data acquisition unit 14 is specifically used to collect the original image dataset of the fully-mechanized coal mining face. The original image dataset includes the initial image data of the fully-mechanized coal mining face under different working conditions and different lighting conditions; label the unique color label of each instance in the initial image data through an interactive segmentation tool; generate an initial image dataset according to the initial image data and the unique color label of each instance in the initial image data.

[0149] In an embodiment of the present invention, the offline data enhancement unit 15 is specifically used to perform geometric transformation on each initial image data in the initial image dataset to generate enhanced image data. The geometric transformation includes affine transformation, shear transformation, uniaxial perspective transformation, and biaxial perspective transformation; convert the unique color label into a single-channel format label through a palette mode; generate an offline image dataset according to the enhanced image data and the single-channel format label.

[0150] In an embodiment of the present invention, the model training unit 16 is specifically used to perform online data enhancement on the offline image dataset to generate a training image dataset; perform model training on the improved instance segmentation algorithm according to the training image dataset through a preset loss function and optimizer to generate an initial segmentation model.

[0151] In the embodiments of the present invention, the initial segmentation result includes: a target mask, class information, and a confidence score; the instance filtering unit 13 is specifically configured to calculate the intersection over union (IoU) of any two target masks to generate a mask IoU; generate an attenuation coefficient according to a preset temperature parameter and the mask IoU through a preset attenuation function; update the confidence score according to the attenuation coefficient to generate an updated confidence score; and perform instance filtering on the initial segmentation result according to the updated confidence score through a preset instance filtering threshold to generate an image instance segmentation result.

[0152] In the solution of the embodiments of the present invention, target image data of a fully-mechanized coal mining face is obtained; instance segmentation is performed on the target image data through a pre-constructed initial segmentation model to generate an initial segmentation result; instance filtering is performed on the initial segmentation result through a preset region-based non-maximum suppression algorithm to generate an image instance segmentation result. By optimizing the features of devices such as the support rib protection plate, it is not only possible to accurately separate adjacent support rib protection plates, but also to simultaneously identify other devices such as the scraper conveyor, shearer drum, ground coal, trough side, track, cable trough, etc., and to distinguish different instances of the same type of device, especially to distinguish overlapping adjacent support rib protection plates, achieving instance-level precise positioning ability, improving segmentation accuracy, reducing computational complexity, and meeting the requirements of real-time monitoring.

[0153] The system, device, module, or unit described in the above embodiments can be specifically implemented by a computer chip or an entity, or by a product with a certain function. A typical implementation device is a computer device. Specifically, the computer device can be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0154] The embodiments of the present invention provide a computer device, including a memory and a processor. The memory is used to store information including program instructions, and the processor is used to control the execution of the program instructions. When the program instructions are loaded and executed by the processor, the steps of the embodiments of the above image instance segmentation method for a fully-mechanized coal mining face are implemented. For specific descriptions, reference can be made to the embodiments of the above image instance segmentation method for a fully-mechanized coal mining face.

[0155] Next, refer to Figure 7 , which shows a schematic structural diagram of a computer device 600 suitable for implementing the embodiments of the present application.

[0156] As Figure 7As shown, the computer device 600 includes a central processing unit (CPU) 601, which can perform various appropriate operations and processes according to the programs stored in the read-only memory (ROM) 602 or the programs loaded from the storage section 608 into the random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the computer device 600 are also stored. The CPU 601, ROM 602, and RAM 603 are connected to each other via a bus 604. The input / output (I / O) interface 605 is also connected to the bus 604.

[0157] The following components are connected to the I / O interface 605: an input section 606 including a keyboard, a mouse, etc.; an output section 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc. and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, a modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed so that a computer program read from it can be installed in the storage section 608 as needed.

[0158] Specifically, according to an embodiment of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, an embodiment of the present invention includes a computer program product, which includes a computer program tangibly embodied on a machine-readable medium, and the computer program includes program code for performing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from the network via the communication section 609, and / or installed from the removable medium 611.

[0159] A computer-readable medium includes permanent and non-permanent, removable and non-removable media that can implement information storage by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette tapes, magnetic disk storage or other magnetic storage devices, or any other non-transitory medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media, such as modulated data signals and carrier waves.

[0160] For convenience of description, when describing the above devices, they are described separately as various units according to their functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0161] The present invention is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in Figure 1 one or more of the flows Figure 1 or blocks or combinations of blocks.

[0162] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including instruction means that implement the functions specified in Figure 1 one or more of the flows Figure 1 or blocks or combinations of blocks.

[0163] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one process or multiple processes and / or blocks Figure 1 one process or multiple processes and / or blocks Figure 1 steps of the functions specified in one block or multiple blocks.

[0164] It should also be noted that the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, commodity or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or also includes elements inherent in such process, method, commodity or device. Without further limitation, an element defined by the statement "comprising one..." does not exclude the presence of additional identical elements in the process, method, commodity or device comprising the said element.

[0165] In the technical solution of this application, the acquisition, storage, use, processing, etc. of data all comply with the relevant provisions of national laws and regulations.

[0166] It should be noted that in the embodiments of this application, some industry-existing solutions such as certain software, components, models, etc. may be mentioned. They should be regarded as exemplary, and their purpose is only to illustrate the feasibility in the implementation of the technical solution of this application, but it does not mean that the applicant has already or necessarily used this solution.

[0167] Those skilled in the art should understand that the embodiments of this application can be provided as a method, a system or a computer program product. Therefore, this application can take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware aspects. Moreover, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0168] This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This application can also be practiced in a distributed computing environment where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0169] Each embodiment in this specification is described in a progressive manner. For the same or similar parts among the embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and for the relevant parts, reference can be made to the partial description of the method embodiment.

[0170] The above are only the embodiments of the present application and are not intended to limit the present application. For those skilled in the art, various changes and modifications can be made to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the scope of the claims of the present application.

Claims

1. An image instance segmentation method for fully-mechanized mining faces, characterized in that The method includes: Obtaining target image data of a fully-mechanized mining face; Performing instance segmentation on the target image data through a pre-constructed initial segmentation model to generate an initial segmentation result; Performing instance filtering on the initial segmentation result through a preset region-based non-maximum suppression algorithm to generate an image instance segmentation result.

2. The image instance segmentation method for fully-mechanized mining face according to claim 1, wherein It further includes: Obtaining an initial image dataset of a fully-mechanized mining face, where the initial image dataset includes multiple pieces of initial image data and a unique color label for each instance in each piece of initial image data; Performing offline data augmentation on the initial image dataset to generate an offline image dataset; Training a model of an improved instance segmentation algorithm through the offline image dataset to construct the initial segmentation model, where the improved instance segmentation algorithm includes a residual network and a feature pyramid network.

3. The image instance segmentation method for fully-mechanized coal mining face according to claim 2, wherein The obtaining of the initial image dataset of a fully-mechanized mining face includes: Collecting a raw image dataset of a fully-mechanized mining face, where the raw image dataset includes initial image data of the fully-mechanized mining face under different working conditions and different lighting conditions; Annotating a unique color label for each instance in the initial image data through an interactive segmentation tool; Generating the initial image dataset according to the initial image data and the unique color label for each instance in the initial image data.

4. The method for image instance segmentation for fully-mechanized coal mining face according to claim 2, wherein The performing of offline data augmentation on the initial image dataset to generate an offline image dataset includes: Performing geometric transformation on each piece of initial image data in the initial image dataset to generate enhanced image data, where the geometric transformation includes affine transformation, shear transformation, uniaxial perspective transformation, and biaxial perspective transformation; Converting the format of the unique color label through a palette mode to generate a single-channel format label; Generating the offline image dataset according to the enhanced image data and the single-channel format label.

5. The image instance segmentation method for fully-mechanized coal mining face according to claim 2, wherein The training of a model of an improved instance segmentation algorithm through the offline image dataset to construct the initial segmentation model includes: Performing online data augmentation on the offline image dataset to generate a training image dataset; Training a model of the improved instance segmentation algorithm according to the training image dataset through a preset loss function and optimizer to generate the initial segmentation model.

6. The image instance segmentation method for fully-mechanized mining face according to claim 1, wherein The initial segmentation result includes: a target mask, class information, and a confidence score; The performing of instance filtering on the initial segmentation result through a preset region-based non-maximum suppression algorithm to generate an image instance segmentation result includes: Calculating the intersection over union of any two of the target masks to generate a mask intersection over union; Generating an attenuation coefficient according to a preset temperature parameter and the mask intersection over union through a preset attenuation function; Updating the confidence score according to the attenuation coefficient to generate an updated confidence score; Performing instance filtering on the initial segmentation result according to the updated confidence score through a preset instance filtering threshold to generate an image instance segmentation result.

7. An image instance segmentation device for a fully mechanized coal mining face, characterized in that The device includes: A target data acquisition unit for obtaining target image data of a fully-mechanized mining face; An instance segmentation unit for performing instance segmentation on the target image data through a pre-constructed initial segmentation model to generate an initial segmentation result; An instance filtering unit for performing instance filtering on the basis of the initial segmentation result through a preset region-based non-maximum suppression algorithm to generate an image instance segmentation result.

8. The image instance segmentation device for fully-mechanized mining face according to claim 7, wherein It further includes: An initial data acquisition unit for acquiring an initial image data set of a fully-mechanized mining face, where the initial image data set includes multiple pieces of initial image data and a unique color label for each instance in each piece of initial image data; An offline data augmentation unit for performing offline data augmentation on the initial image data set to generate an offline image data set; A model training unit for training a model of an improved instance segmentation algorithm through the offline image data set to construct the initial segmentation model, where the improved instance segmentation algorithm includes a residual network and a feature pyramid network.

9. The image instance segmentation device for fully-mechanized mining face according to claim 8, characterized in that, The initial data acquisition unit is specifically configured to collect a raw image data set of a fully-mechanized mining face, where the raw image data set includes initial image data of the fully-mechanized mining face under different working conditions and different lighting conditions; label a unique color label for each instance in the initial image data through an interactive segmentation tool; generate the initial image data set according to the initial image data and the unique color label for each instance in the initial image data.

10. The image instance segmentation device for fully-mechanized mining face according to claim 8, characterized in that, The offline data augmentation unit is specifically configured to perform geometric transformation on each piece of initial image data in the initial image data set to generate augmented image data, where the geometric transformation includes affine transformation, shear transformation, uniaxial perspective transformation, and biaxial perspective transformation; convert the format of the unique color label through a palette mode to generate a single-channel format label; generate the offline image data set according to the augmented image data and the single-channel format label.

11. The image instance segmentation device for fully-mechanized mining face according to claim 8, wherein The model training unit is specifically configured to perform online data augmentation on the offline image data set to generate a training image data set; train a model of the improved instance segmentation algorithm according to the training image data set through a preset loss function and optimizer to generate the initial segmentation model.

12. The image instance segmentation device for fully-mechanized mining face according to claim 7, characterized in that, The initial segmentation result includes: a target mask, class information, and a confidence score; The instance filtering unit is specifically configured to calculate the intersection over union of any two of the target masks to generate a mask intersection over union; generate an attenuation coefficient according to a preset temperature parameter and the mask intersection over union through a preset attenuation function; update the confidence score according to the attenuation coefficient to generate an updated confidence score; perform instance filtering on the initial segmentation result according to the updated confidence score through a preset instance filtering threshold to generate an image instance segmentation result.

13. A computer-readable medium having a computer program stored thereon, characterized in that, When the program is executed by a processor, it implements the image instance segmentation method for a fully-mechanized mining face according to any one of claims 1 to 6.

14. A computer device, comprising a memory and a processor, the memory being used for storing information including program instructions, and the processor being used for controlling the execution of the program instructions, characterized in that, When the program instructions are loaded and executed by a processor, it implements the image instance segmentation method for a fully-mechanized mining face according to any one of claims 1 to 6.

15. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by a processor, the method for image instance segmentation for fully-mechanized coal mining face according to any one of claims 1 to 6 is implemented.

Images