Training method of coal mine underground image processing model and image processing method

Abstract

The application relates to the technical field of image processing, and proposes a training method of an underground coal mine image processing model and an image processing method. The training method comprises the following steps: determining the resolution of sample coal mine images in a coal mine scene database; constructing a sub-model sequence according to the resolution, wherein each sub-model comprises an encoder and a decoder; independently training the sub-model sequence to obtain a target sub-model sequence; constructing a first coal mine image processing model based on the target sub-model sequence; and fine-tuning the first model to obtain a second coal mine image processing model. The second model optimizes the fusion effect of the sub-models, improves the multi-scale feature representation capability in a complex coal mine scene, and realizes efficient image segmentation under low illumination and high dust interference conditions, and has the technical advantages of fast model convergence and sufficient feature fusion.

Description

Technical Field

[0001] This application belongs to the field of image processing technology, specifically relating to a training method and image processing method for an underground coal mine image processing model. Background Technology

[0002] Current multi-scale segmentation methods for coal and rock images in complex underground coal mine scenarios suffer from insufficient feature extraction, high initialization sensitivity, and severe structural redundancy. Specifically, traditional U-Net networks struggle to fully exploit the multi-scale features of coal and rock images during end-to-end training, random initialization of deep network parameters easily leads to difficulties in overall architecture convergence, and the number of conventional network parameters exceeds the carrying capacity of underground equipment. Therefore, a coal mine image processing method that can solve the above problems is urgently needed. Summary of the Invention

[0003] This application aims to solve at least one technical problem in the related art.

[0004] Therefore, the first objective of this application is to propose a training method for an underground coal mine image processing model to improve the representation ability of multi-scale features in complex scenes and enhance image processing performance.

[0005] The second objective of this application is to propose an image processing method.

[0006] The third objective of this application is to propose a training device for an image processing model in underground coal mines.

[0007] The fourth objective of this application is to provide an image processing apparatus.

[0008] The fifth objective of this application is to provide an electronic device.

[0009] The sixth objective of this application is to provide a computer-readable storage medium.

[0010] The seventh objective of this application is to provide a computer program product.

[0011] To achieve the above objectives, the first aspect of this application proposes a training method for an underground coal mine image processing model, comprising the following steps: Determine the resolution of sample coal mine images in the coal mine scene database; Based on the resolution of the sample coal mine images, a sequence of sub-models is constructed, wherein each sub-model in the sequence includes an encoder and a decoder; The target sub-model sequence is obtained by independently training each sub-model in the sub-model sequence. Based on the target sub-model sequence, a first coal mine image processing model is constructed; The first coal mine image processing model was fine-tuned and trained to obtain the second coal mine image processing model.

[0012] To achieve the above objectives, a second aspect of this application provides an image processing method, comprising: Acquire the target coal mine image to be processed; Determine the resolution of the target coal mine image and select a second coal mine image processing model that matches the resolution; The target coal mine image is input into the second coal mine image processing model, and the prediction result of the target coal mine image is output. The second coal mine image processing model is a model obtained using the training method described in the first aspect embodiment.

[0013] To achieve the above objectives, a third aspect of this application provides a training apparatus for a coal mine image processing model, comprising: The first acquisition module is used to determine the resolution of sample coal mine images in the coal mine scene database; The second acquisition module is used to construct a sub-model sequence based on the resolution of the sample coal mine image, wherein each sub-model in the sub-model sequence includes an encoder and a decoder; The third acquisition module is used to independently train the sub-models in the sub-model sequence to obtain the target sub-model sequence. The fourth acquisition module is used to construct a first coal mine image processing model based on the target sub-model sequence; The fine-tuning module is used to fine-tune the first coal mine image processing model to obtain the second coal mine image processing model.

[0014] To achieve the above objectives, a fourth aspect of this application provides an image processing apparatus, comprising: The first acquisition module is used to acquire the target coal mine image to be processed; The second acquisition module is used to determine the resolution of the target coal mine image and select a second coal mine image processing model that matches the resolution. The prediction module is used to input the target coal mine image into the second coal mine image processing model and output the prediction result of the target coal mine image; wherein, the second coal mine image processing model is a model obtained by using the training method described in the first aspect embodiment.

[0015] To achieve the above objectives, a fifth aspect of this application provides an electronic device, including: a processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in the first aspect embodiment.

[0016] To achieve the above objectives, a sixth aspect of this application provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, are used to implement the method described in the first aspect embodiment.

[0017] To achieve the above objectives, a seventh aspect of this application provides a computer program product, including a computer program that, when executed by a processor, implements the method described in the first aspect embodiment.

[0018] This application provides a training method and an image processing method for an underground coal mine image processing model. The specific steps are as follows: Determine the resolution: Determine the resolution parameters by analyzing coal mine images in the coal mine scene database.

[0019] Constructing a sub-model sequence: The number of sub-models in the sub-model sequence is dynamically determined based on the image resolution, thereby generating a suitable sub-model sequence.

[0020] Independent training of sub-models: Each sub-model is trained independently to form a sequence of target sub-models.

[0021] Constructing the first coal mine image processing model: The first coal mine image processing model is constructed based on the target sub-model sequence. This model contains multiple trained sub-models.

[0022] Fine-tuning training: The first coal mine image processing model is fine-tuned and trained to generate the second coal mine image processing model.

[0023] The first coal mine image processing model includes multiple trained sub-models, which can effectively improve the model's multi-scale feature representation capability in complex coal mine scenes. The second coal mine image processing model, obtained by fine-tuning the first coal mine image processing model, allows for better fusion of multiple sub-models, improving segmentation accuracy under low illumination and high dust interference conditions. It has the technical advantages of fast model convergence and sufficient feature fusion.

[0024] Additional aspects and advantages of this application will be partly presented in the description below, and partly demonstrated through the implementation of this application. Attached Figure Description

[0025] The above aspects and additional advantages of this application will become clearer and easier to understand from the description of the embodiments in conjunction with the accompanying drawings. The drawings include: Figure 1A flowchart illustrating a training method for an underground coal mine image processing model provided in an embodiment of this application; Figure 2 A flowchart illustrating another method for training an underground coal mine image processing model provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of a sub-model sequence provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of the first coal mine image processing model provided in the embodiments of this application; Figure 5 A flowchart illustrating an image processing method provided in an embodiment of this application; Figure 6 A schematic diagram of the structure of a training device for an underground coal mine image processing model provided in an embodiment of this application; Figure 7 This is a schematic diagram of the structure of an image processing device provided in an embodiment of this application. Detailed Implementation

[0026] Embodiments of this application will be described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same function. The embodiments described with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0027] The following describes, with reference to the accompanying drawings, a training method and an image processing method for an underground coal mine image processing model according to embodiments of this application.

[0028] Figure 1 This is a schematic flowchart illustrating a training method for an underground coal mine image processing model provided in an embodiment of this application. Figure 1 As shown, the method includes the following steps: S101, Determine the resolution of sample coal mine images in the coal mine scene database.

[0029] Optionally, the resolution of the sample coal mine images in the coal mine scene database can be obtained by viewing the images using image viewing tools or image management tools. Alternatively, the resolution of the sample coal mine images can be obtained by performing spectral energy distribution analysis on the sample coal mine images using methods such as Fourier transform.

[0030] In the current technological field, resolution refers to the density of detail information contained in an image, usually quantified by the number of pixels (such as resolution parameters), and is a key indicator affecting image clarity and analytical accuracy. In coal mine applications, resolution directly affects the image processing system's ability to identify key elements, such as the accurate capture and analysis of equipment operating status, personnel work behavior, and abnormal environmental features.

[0031] S102, construct a sub-model sequence based on the resolution of the sample coal mine images.

[0032] In some embodiments, the sub-model sequence may include one or more sub-models, wherein each sub-model in the sub-model sequence includes an encoder and a decoder; the input to each sub-model is image data that may be downsampled at different scales.

[0033] In some embodiments, if the sample coal mine image has a low resolution, a shorter sub-model sequence is determined; correspondingly, if the sample coal mine image has a high resolution, a longer sub-model sequence is determined.

[0034] For example, if the resolution of the sample coal mine image is lower than 512×512, then interpolation is used to perform K=3 levels of downsampling; if the resolution of the sample coal mine image is higher than 512×512, then interpolation is used to perform K=4 levels of downsampling; the construction of the sub-model corresponds to the K-level downsampling, that is, the number of sub-models in the sub-model sequence is K+1.

[0035] S103, train the sub-models in the sub-model sequence independently to obtain the target sub-model sequence.

[0036] In some embodiments, a set of sample images can be obtained to train the sub-model. For example, the sample images are input into the sub-model to obtain the prediction results of the sub-model. The loss function is determined based on the prediction results and the standard results of the sample images. The model parameters are adjusted until the loss function converges, and the training of the sub-model is completed, thus obtaining the target sub-model.

[0037] Each sub-model is trained independently to obtain the corresponding target sub-model, thus forming a sequence of target sub-models.

[0038] S104. Construct the first coal mine image processing model based on the target sub-model sequence.

[0039] Optionally, the output of the target model sequence can be connected to the fusion layer, which then splices and fuses the target sub-model sequences.

[0040] Optionally, a decoder can be connected after the fusion layer as a global decoder for the first coal mine image processing model. In some embodiments, the decoder may include a channel dimensionality reduction layer to reduce the image to the same level as the original channel dimension.

[0041] Optionally, a channel expansion layer may be included before the target sub-model sequence to input the channel-expanded image into each target sub-model. A channel dimensionality reduction layer may be included after each target sub-model to input the output of each target sub-model into a fusion layer for fusion. Further, the output is performed after the decoder processing.

[0042] The image processing model for the first coal mine includes, but is not limited to, a target sub-model sequence, a fusion layer, a channel expansion layer, and a decoder.

[0043] S105, fine-tune and train the first coal mine image processing model to obtain the second coal mine image processing model.

[0044] Alternatively, the original resolution image can be used as input for end-to-end fine-tuning of the model.

[0045] Optionally, the first coal mine image processing model can be fine-tuned using an Adaptive Moment Estimation (Adam) optimizer and a cosine annealing learning rate decay strategy to obtain a pre-trained second coal mine image processing model.

[0046] This embodiment includes the following steps: Determine the resolution: Determine the resolution of coal mine images by analyzing the coal mine scene database.

[0047] Constructing a sub-model sequence: Based on the resolution, the number of sub-models in the sub-model sequence is dynamically determined, thereby generating a suitable sub-model sequence.

[0048] Independent training of sub-models: Each sub-model in the sub-model sequence is trained independently to form the target sub-model sequence.

[0049] Constructing the first model: Constructing the first coal mine image processing model based on the target sub-model sequence. This model contains multiple trained sub-models.

[0050] Fine-tuning training: The first model is fine-tuned to generate the second coal mine image processing model.

[0051] Through the above steps, the first coal mine image processing model effectively improves the multi-scale feature representation capability under complex coal mine scenes. The second coal mine image processing model optimizes the fusion effect of multiple sub-models, improves the segmentation accuracy under low illumination and high dust interference conditions, and has the technical advantages of fast model convergence and sufficient feature fusion.

[0052] Figure 2 This is a schematic flowchart illustrating another method for training an underground coal mine image processing model provided in an embodiment of this application. Figure 2As shown, the method may include the following steps: S201, Determine the resolution of sample coal mine images in the coal mine scene database.

[0053] In this application embodiment, the implementation method of step S201 can be implemented in any of the various embodiments of this disclosure, and no limitation is made here, nor will it be described in detail.

[0054] S202, construct a sub-model sequence based on the resolution of the sample coal mine images.

[0055] In some embodiments, the number of encoder modules included in the sub-model is initialized to 2.

[0056] Optionally, the m-th sub-model including n encoders and the m-th sub-model including n+1 encoders can be validated based on the validation set, and the validation intersection over union (IoU) ratio of the two can be obtained respectively; that is, for the m-th sub-model, its validation IoU under the n encoder configuration and its validation IoU under the n+1 encoder configuration are determined.

[0057] In this embodiment, we verify that IoU can be used to reflect the performance of the sub-model. When the number of encoders is large, the ability to extract features can be improved, but the computational cost will also increase accordingly. Therefore, determining the optimal number of encoders based on task complexity and data scale can avoid the risk of overfitting and reduce computational cost.

[0058] In some embodiments, the intersection-to-union ratio (IoU) difference between the m-th sub-model including n+1 encoders and the m-th sub-model including n encoders can be determined; when the IoU difference is greater than or equal to a preset IoU threshold, the number of encoders in the m-th sub-model is determined to be n+1.

[0059] For example, the verification IoU of the m-th sub-model including n+1 encoders is denoted as the first IoU, and the verification IoU of the m-th sub-model including n encoders is denoted as the second IoU. The difference between the first IoU and the second IoU is calculated. If the difference is greater than or equal to a preset IoU threshold (e.g., 0.05), it is determined that the configuration of the n+1 encoders corresponding to the first IoU is better, thus determining that the number of encoders in the m-th sub-model is n+1. Conversely, if the difference is less than the IoU threshold, it is determined that the configuration of the n encoders corresponding to the second IoU is better, thus determining that the number of encoders in the m-th sub-model is n. Each sub-model is processed sequentially in the above manner until the number of encoders in each sub-model is determined.

[0060] In some embodiments, the number of decoders in different sub-models can be the same, for example, the number of decoders can be 1.

[0061] In some embodiments, the encoder and decoder within the same sub-model have the same number of channels. The number of channels corresponding to the m-th sub-model is determined according to the following formula: , in This represents the number of channels corresponding to the m-th sub-model.

[0062] In some embodiments, the encoder and decoder in each sub-model use the same network module, such as a 3×3 convolutional module, or a residual module, or an inverse residual module, or a lightweight Transformer module, etc. The m-th sub-model uses... The model input is downsampled by a factor of 1.5. To reduce the complexity of the sub-model, the decoder of the sub-model uses a method that... A single upsampling operation is performed, multiplied by a factor of 1. The upsampling method is either bilinear interpolation or transposed convolution, restoring the image to its original size. For details, please refer to [link / reference]. Figure 3 Up in operate.

[0063] In this application embodiment, the implementation method of step S202 can be implemented in any of the various embodiments of this disclosure, and no limitation is made here, nor will it be described in detail.

[0064] S203, for the m-th sub-model in the sub-model sequence, determine the corresponding input coal mine image and learning rate based on its sequence number.

[0065] Optionally, the downsampling rate corresponding to the m-th sub-model can be determined first based on the index of the m-th sub-model; where the sub-model index refers to 1, 2, 3, ..., m, ..., K+1, and K+1 is the total number of all sub-models; for the sub-model with index m, the downsampling rate corresponding to the m-th sub-model can be expressed as... .

[0066] Furthermore, the sample coal mine image can be downsampled according to the downsampling rate corresponding to the m-th sub-model to obtain the input coal mine image corresponding to the m-th sub-model. That is, the sample coal mine image is downsampled according to the downsampling rate corresponding to each sub-model to obtain the input coal mine image corresponding to each sub-model.

[0067] Optionally, the formula for obtaining the learning rate of the m-th sub-model can be:

[0068] in, This represents the learning rate corresponding to the m-th sub-model.

[0069] In this application, sub-models with different indices employ differentiated learning rate configurations. The learning rate is a key hyperparameter in machine learning, used to control the step size of model parameter updates during gradient descent, directly affecting the magnitude of parameter adjustments in each iteration. To meet the training requirements of each sub-model, this application adopts an exponential decay strategy: as the sub-model index m increases, the corresponding learning rate decreases exponentially. This design balances training efficiency and stability. Initial phase: Use a high initial learning rate to accelerate model convergence to a local optimum; In the later stages: through the exponential decay mechanism, the learning rate is gradually reduced to a lower level, effectively avoiding the risk of gradient oscillation or model divergence caused by an excessively high learning rate, while improving the precision of parameter tuning.

[0070] For example, the first sub-model quickly captures the salient features of the data distribution with a learning rate of 0.1; while subsequent sub-models are progressively optimized based on the decaying learning rate, ultimately achieving more stable feature learning and parameter convergence in complex scenarios such as low light and high dust. This strategy not only accelerates the convergence speed of the model in the initial stage, but also significantly improves the model's generalization ability through fine-tuning of the low learning rate in the later stages.

[0071] S204, based on the input coal mine image and the learning rate, train the m-th sub-model until the training ends and the target sub-model sequence is obtained.

[0072] In some embodiments, the training process of the m-th sub-model includes the following steps: Input a coal mine image into the m-th sub-model and obtain the prediction results.

[0073] The loss function is determined based on the difference between the predicted results and the standard results.

[0074] The gradient of the loss function with respect to the model parameters is calculated and passed layer by layer using the chain rule.

[0075] The optimizer updates the model parameters according to the learning rate.

[0076] Monitor model convergence and periodically check and validate IoU to avoid overfitting.

[0077] Through multiple iterations of training, the training continues until the end condition is met: verify that the IoU is greater than or equal to 0.85.

[0078] The target sub-model is finally obtained.

[0079] In this embodiment, K+1 sub-models are trained independently to obtain the target sub-models corresponding to the K+1 sub-models, which is the target sub-model sequence.

[0080] S205. Construct the first coal mine image processing model based on the target sub-model sequence.

[0081] In some embodiments, the first coal mine image processing model may include: a channel expansion layer, a global encoder, a fusion layer, and a global decoder; wherein, the channel expansion layer is connected to the encoder of the first sub-model in the target sub-model sequence; the global encoder is obtained by concatenating the encoders of the sub-models in the target sub-model sequence; the fusion layer includes a channel dimensionality reduction layer, a fusion layer, and a feature enhancement layer associated with the encoder of each sub-model; the output of the channel dimensionality reduction layer is fused into the fusion layer, and the output of the concatenated fusion layer is enhanced into the feature enhancement layer; the global decoder includes at least one channel dimensionality reduction layer, and the output of the feature enhancement layer is decoded into the global decoder, wherein the fusion weights corresponding to each channel dimensionality reduction layer are the same; the overall architecture of the first coal mine image processing model is as follows. Figure 4 As shown.

[0082] In some embodiments, the first coal mine image processing model includes a channel expansion layer, a global encoder, a fusion layer, and a global decoder. The specific structure is as follows: Channel extension layer: Connected to the encoder of the first sub-model in the target sub-model sequence.

[0083] Global encoder: formed by concatenating the sub-model encoders in the target sub-model sequence.

[0084] Fusion layer: contains the channel dimensionality reduction layer, fusion layer, and feature enhancement layer associated with each sub-model encoder.

[0085] Global decoder: includes at least one channel dimensionality reduction layer; the output of the feature enhancement layer is input to the global decoder for decoding, where the fusion weights corresponding to each channel dimensionality reduction layer are the same.

[0086] The fusion and enhancement process is specifically manifested as follows: the output of the channel dimensionality reduction layer is input to the fusion layer for fusion; the output of the fusion layer is spliced ​​and then input to the feature enhancement layer for enhancement.

[0087] The overall architecture of the image processing model for the first coal mine is as follows: Figure 4 As shown.

[0088] In some embodiments, if multiple channel dimensionality reduction layers exist, a splicing and fusion layer is further included after the multiple channel dimensionality reduction layers, and the fusion weights corresponding to each channel dimensionality reduction layer are the same. For example, if there are 5 channel dimensionality reduction layers, the fusion weight of each channel dimensionality reduction layer is 1 / 5.

[0089] S206, Fine-tune and train the first coal mine image processing model to obtain the second coal mine image processing model.

[0090] Optionally, the model parameters of the first coal mine image processing model can be initialized. Initialization includes retaining the weights of each encoder in the global encoder and configuring the weights of the global decoder based on a Gaussian distribution. In this embodiment, the fine-tuning process follows a "large-to-small" principle, adjusting the Gaussian distribution parameters of the global decoder sequentially. Specifically, this includes gradually setting the mean and variance of the Gaussian distribution according to levels or modules to achieve more reasonable parameter initialization and subsequent fine-tuning effects.

[0091] In some embodiments, the fine-tuning process of the global encoder includes the following steps: Encoding Stage Division: Based on structural location, the encoders in the global encoder are divided into two encoding stages. For example, when the global encoder includes four encoders, the third and fourth encoders can be used as the first encoding stage, and the first and second encoders as the second encoding stage.

[0092] Freeze the first-stage weights: In the initial stage, freeze the weight parameters of the first encoding stage to prevent them from being updated in subsequent training.

[0093] Fine-tuning the second stage: Fine-tuning the second encoding stage by optimizing the model parameters of this stage through backpropagation.

[0094] Response fine-tuning ends: After the fine-tuning in the second encoding stage is completed, the weight parameters for that stage are frozen.

[0095] Fine-tuning the first stage: Subsequently, the first encoding stage is fine-tuned to further optimize its parameters.

[0096] Training complete: The training process ends when the preset fine-tuning termination conditions are met (such as reaching the maximum number of iterations or verifying performance convergence), and the final second coal mine image processing model is obtained.

[0097] This method, through a phased fine-tuning strategy, can effectively avoid gradient conflicts caused by adjusting all parameters simultaneously and significantly reduce computational costs.

[0098] It should be noted that all model adjustments in this embodiment use the Adam optimizer, employ a cosine annealing learning rate decay strategy, and utilize a hybrid loss function. in Here, W represents the adjustment coefficients for different losses, and W represents the network architecture weights. This is the classification loss, used to address the imbalance between positive and negative samples in classification tasks.

[0099] The coal mine image processing model construction method in this embodiment includes the following steps: a. Sub-model sequence generation Based on the comparison between the target image resolution and a preset threshold, the number of sub-models is dynamically determined, and a sub-model sequence is generated. The validation IoU is calculated using a validation set for different numbers of encoders. Based on the IoU threshold, the optimal number of encoders is selected to achieve optimized resource allocation.

[0100] b. Differentiated training strategy The learning rate is allocated according to the sub-model number m (e.g., η = η0 × 10). -m ) and input image; The input image is downsampled to different degrees according to the sequence number (e.g., when m=1, the downsampling rate is 0.5; when m=3, the downsampling rate is 0.25). The sub-models are trained independently based on the learning rate and downsampled images until the convergence condition is met (e.g., verifying IoU ≥ 0.85), and the target sub-model sequence is output.

[0101] c. Multi-level model construction The first coal mine image processing model is constructed based on the target sub-model sequence, integrating the channel expansion layer, fusion layer and global encoder; Cross-scale feature fusion is achieved through feature enhancement layers (e.g.) Figure 4 The architecture shown enhances the ability to represent features in complex scenarios.

[0102] d. Phased fine-tuning and optimization The global encoder is divided into two stages (e.g., the first two layers are stage A, and the last two layers are stage B). Alternately execute weight freeze and fine-tuning: freeze phase A weights, fine-tuning phase B; freeze phase B weights, fine-tuning phase A. After the fine-tuning termination condition is met, the second coal mine image processing model is output, thereby effectively reducing the amount of computation during the training process while avoiding gradient conflicts.

[0103] The coal mine image processing model construction method of this embodiment has the following technical advantages: reducing redundant computing resource consumption through resolution adaptive mechanism; avoiding gradient conflicts caused by synchronous parameter adjustment through phased fine-tuning strategy; and improving the generalization of the model in low-light and high-dust scenarios through downsampling input and exponential decay learning rate in synergy.

[0104] Figure 5 A flowchart of an image processing method provided in this application embodiment, the method including the following steps: S501, acquire the target coal mine image to be processed.

[0105] Optionally, the target coal mine image can be an image acquired in real time, or an image acquired in advance and stored in a storage unit, and the target coal mine image can be used for image processing and analysis.

[0106] S502, determine the resolution of the target coal mine image, and select a second coal mine image processing model that matches the resolution.

[0107] Optionally, a mapping relationship between the resolution and the second coal mine image processing model can be determined. Based on this resolution, the mapping relationship is queried to obtain the second coal mine image processing model that matches the resolution.

[0108] The number of sub-models in the second coal mine image processing model varies with the resolution. For example, when the resolution is lower than 512×512, K=3, and a second coal mine image processing model with K+1=4 sub-models is used; when the resolution is higher than 512×512, K=4, and a second coal mine image processing model with K+1=5 sub-models is used.

[0109] S503: Input the target coal mine image into the second coal mine image processing model and output the prediction result of the target coal mine image.

[0110] The second coal mine image processing model is a model obtained by using the training method described in the first or second aspect embodiment.

[0111] The target coal mine image is input into the second coal mine image processing model, which performs image processing and analysis to obtain the prediction result. This prediction result represents the optimal multi-scale segmentation result of the coal mine image.

[0112] In this embodiment, after determining the target coal mine image to be processed, a second coal mine image processing model matching the resolution of the target coal mine image is selected. This ensures the most suitable model is matched to reduce the consumption of redundant computing resources and lower computational costs. After obtaining the corresponding second coal mine image processing model, the target coal mine image is input into the model, and its prediction results are output, improving the ability to analyze multi-scale features and ensuring high-precision output of prediction results.

[0113] This embodiment includes the following steps: Target image determination: Determine the target coal mine image to be processed.

[0114] Model adaptation selection: Based on the resolution parameters of the target coal mine image, a corresponding second coal mine image processing model is selected to match the most suitable model configuration. This selection mechanism can reduce redundant resource consumption and lower computational costs.

[0115] Image processing execution: Input the target coal mine image into the selected second coal mine image processing model for processing, and output the prediction results.

[0116] The above process effectively improves the accuracy of multi-scale feature analysis and ensures the reliability of prediction results.

[0117] To implement the above embodiments, this application also proposes a training device for an underground coal mine image processing model.

[0118] Figure 6 This is a schematic diagram of the structure of a training device for an underground coal mine image processing model provided in an embodiment of this application. Figure 6 As shown, the training device 600 for the underground coal mine image processing model includes: The first acquisition module 601 is used to determine the resolution of sample coal mine images in the coal mine scene database; The second acquisition module 602 is used to construct a sub-model sequence based on the resolution of the sample coal mine image, wherein each sub-model in the sub-model sequence includes an encoder and a decoder. The third acquisition module 603 is used to independently train the sub-models in the sub-model sequence to obtain the target sub-model sequence. The fourth acquisition module 604 is used to construct the first coal mine image processing model based on the target sub-model sequence; The fine-tuning module 605 is used to fine-tune the first coal mine image processing model to obtain the second coal mine image processing model.

[0119] In some implementations, the third acquisition module 603 is configured to perform the following operations: For the m-th sub-model in the sub-model sequence, the corresponding input coal mine image and learning rate are determined based on its sequence number; The sub-model is iteratively trained using the input coal mine image and the learning rate until convergence, generating a target sub-model sequence.

[0120] In some implementations, the learning rate for the m-th sub-model is determined using the following formulas, including:

[0121] in, This represents the learning rate corresponding to the m-th sub-model.

[0122] In some implementations, device 600 also includes: Based on the validation set, the m-th sub-model containing n encoders and the m-th sub-model containing n+1 encoders are validated respectively, and the validation intersection-union ratio (IoU) of the two is obtained. Calculate the IoU difference between the sub-model containing n+1 encoders and the sub-model containing n encoders; When the IoU difference is greater than or equal to the preset IoU threshold, the number of encoders for the m-th sub-model is determined to be n+1.

[0123] In some implementations, the encoder and decoder within the same sub-model have the same number of channels. The number of channels corresponding to the m-th sub-model is determined by the following formula:

[0124] in This represents the number of channels corresponding to the m-th sub-model.

[0125] In some implementations, the first coal mine image processing model includes a channel expansion layer, a global encoder, a fusion layer, and a global decoder, wherein: The channel extension layer is connected to the encoder of the first sub-model in the target sub-model sequence; The global encoder is formed by sequentially connecting the encoders of each sub-model in the target sub-model sequence. The fusion layer includes a channel dimensionality reduction layer, a feature fusion layer, and a feature enhancement layer associated with each sub-model encoder; The output of the channel dimensionality reduction layer is input to the feature fusion layer to perform feature fusion, and the fused output is further input to the feature enhancement layer for feature enhancement. The global decoder contains at least one channel dimensionality reduction layer, and the output of the feature enhancement layer is input to the global decoder for decoding, wherein the fusion weights corresponding to each channel dimensionality reduction layer remain the same.

[0126] In some implementations, the fine-tuning module 605 is further configured to perform the following operations: The model parameters of the first coal mine image processing model are initialized, including retaining the weight parameters of all encoders in the global encoder and initializing the weights of the global decoder based on Gaussian distribution. The global encoder is divided into two encoding stages based on its structural location; Freeze the weight parameters of the first encoding stage and perform fine-tuning training for the second encoding stage; In response to the completion of the second encoding stage fine-tuning, the weight parameters of the second encoding stage are frozen, and the fine-tuning training of the first encoding stage is started. In response to the satisfaction of the preset fine-tuning termination condition, the second coal mine image processing model is output.

[0127] In some implementations, the decoder for the sub-model uses The single upsampling is multiplied by a factor of 1, and the upsampling method is bilinear interpolation or transposed convolution.

[0128] It should be noted that the explanation of the training method embodiment for the underground coal mine image processing model described above also applies to the training device for the underground coal mine image processing model in this embodiment, and will not be repeated here.

[0129] The coal mine underground image processing model training device provided in this embodiment performs the following operations: Resolution analysis: Determine the resolution characteristics of coal mine images in the coal mine scene database; Sub-model construction: Construct a sequence of sub-models based on the resolution, wherein the number of sub-models in the sequence is positively correlated with the resolution size; Independent training: Perform independent training on each sub-model in the sub-model sequence to generate the target sub-model sequence; Model integration: The first coal mine image processing model is constructed based on the target sub-model sequence. This model integrates multiple trained sub-models, which significantly enhances the multi-scale feature representation capability in complex coal mine scenes. Fine-tuning and optimization: Fine-tuning training is performed on the first coal mine image processing model to generate the second coal mine image processing model; The fine-tuning process improves image segmentation accuracy under low illumination and high dust interference conditions by optimizing the fusion mechanism between sub-models. The overall solution has the core technical advantages of fast model convergence speed and sufficient multi-scale feature fusion.

[0130] To implement the above embodiments, this application also provides an image processing apparatus.

[0131] Figure 7 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of this application. Figure 7 As shown, the image processing apparatus 700 includes the following modules: The first acquisition module 701 is used to acquire the target coal mine image to be processed; The second acquisition module 702 is used to determine the resolution of the target coal mine image and select a second coal mine image processing model that matches the resolution. The prediction module 703 is used to input the target coal mine image into the second coal mine image processing model and output the prediction result of the target coal mine image; wherein, the second coal mine image processing model is a model obtained by using the training method of the first aspect embodiment.

[0132] It should be noted that the foregoing explanation of the image processing method embodiments also applies to the image processing apparatus of this embodiment, and will not be repeated here.

[0133] This embodiment includes the following steps: Target image determination: Determine the target coal mine image to be processed.

[0134] Model adaptation selection: Based on the resolution of the target coal mine image, a corresponding second coal mine image processing model is selected to match the most suitable model configuration. This selection mechanism can reduce redundant resource consumption and lower computational costs.

[0135] Image processing execution: Input the target coal mine image into the selected second coal mine image processing model for processing, and output the prediction results.

[0136] The above process effectively improves the accuracy of multi-scale feature analysis and ensures the reliability of prediction results.

[0137] To implement the above embodiments, this application also provides an electronic device, including: a processor and a memory communicatively connected to the processor; the memory stores computer-executable instructions, and the processor executes the instructions to implement the image processing method described in the foregoing embodiments.

[0138] To implement the above embodiments, this application also provides a non-transitory computer-readable storage medium storing computer-executable instructions thereon. When executed by a processor, the instructions implement the image processing method described in the foregoing embodiments.

[0139] To implement the above embodiments, this application also provides a computer program product comprising executable program code. When the program code runs on a computer or processor, it implements the image processing method described in the foregoing embodiments.

[0140] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in this application all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0141] It should be noted that personal information collected from users should be used for legitimate and reasonable purposes and should not be shared or sold outside of these legitimate uses. Furthermore, such collection / sharing should only be conducted after receiving the user's informed consent, including but not limited to notifying the user to read the user agreement / user notice and sign an agreement / authorization that includes authorization of relevant user information before the user uses the function. In addition, any necessary steps must be taken to protect and safeguard access to such personal information data and ensure that others with access to personal information data comply with their privacy policies and procedures.

[0142] This application is intended to provide an implementation scheme for users to selectively prevent the use or access to their personal information data. Specifically, this disclosure is intended to provide hardware and / or software to prevent or block access to such personal information data. Once personal information data is no longer needed, risks can be minimized by restricting data collection and deleting data. Furthermore, where applicable, such personal information is de-identified to protect user privacy.

[0143] In the foregoing descriptions of the embodiments, the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0144] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0145] Any process or method described in flowcharts or other forms can be interpreted as representing a module, segment, or portion of code. This code contains one or more executable instructions for implementing specific logical functions or process steps. The scope of preferred embodiments of this application also includes other implementations where the order of function execution may not be in the order shown or discussed, including execution in substantially simultaneous or reverse order depending on functional requirements. This should be understood by those skilled in the art.

[0146] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0147] The components of this application can be implemented by hardware, software, firmware, or a combination thereof. In the foregoing embodiments, the methods or steps can be implemented by software or firmware stored in memory and executed by an instruction execution system. If hardware implementation is used, in other embodiments, any one or a combination of the following technologies known in the art can be selected: discrete logic circuits with logic gates (for implementing the logic function of data signals); application-specific integrated circuits (ASICs, containing combinational logic gates with adapted logic functions); programmable gate arrays (PGAs); field-programmable gate arrays (FPGAs), etc.

[0148] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by program instructions and related hardware. The program can be stored in a computer-readable storage medium, and when executed, it includes one or a combination of steps from the method embodiments.

[0149] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0150] Storage media include, but are not limited to, read-only memory, magnetic disks, or optical disks. Although embodiments of this application have been shown and described, it should be understood that these embodiments are exemplary and do not constitute a limitation thereof. Those skilled in the art can make changes, modifications, substitutions, or variations to the embodiments within the scope of this application.

Claims

1. A training method for an underground coal mine image processing model, characterized in that, The method includes: Determine the resolution of sample coal mine images in the coal mine scene database; Based on the resolution of the sample coal mine images, a sub-model sequence is constructed. The downsampling level K is determined according to the resolution; the higher the resolution, the larger K becomes. The number of sub-models in the sub-model sequence is K+1, and the m-th sub-model corresponds to 2... m-1 The input coal mine image is downsampled at a multiple of 1, m=1,2,...,K+1. Each sub-model in the sub-model sequence includes an encoder and a decoder. Based on the validation set, the m-th sub-model with n encoders and the m-th sub-model with n+1 encoders are validated respectively to obtain their validation intersection-over-union (IoU). The IoU difference between the m-th sub-model with n+1 encoders and the m-th sub-model with n encoders is calculated. When the IoU difference is greater than or equal to a preset threshold, the number of encoders in the m-th sub-model is increased from n to n+1, thus determining that the number of encoders in the m-th sub-model is n+1. When the IoU difference is less than the threshold, the number of encoders in the m-th sub-model is determined to be n. The target sub-model sequence is obtained by independently training the sub-models in the sub-model sequence, including the following steps: For the m-th sub-model in the sub-model sequence, determine its corresponding input coal mine image and learning rate based on the index of the m-th sub-model; Based on the input coal mine image and the corresponding learning rate, the m-th sub-model is trained until a preset training termination condition is met, resulting in the target sub-model sequence. The learning rate for the m-th sub-model is determined using the following formula: in, This represents the learning rate corresponding to the m-th sub-model; Based on the target sub-model sequence, a first coal mine image processing model is constructed; The first coal mine image processing model was fine-tuned and trained to obtain the second coal mine image processing model.

2. The method according to claim 1, characterized in that, Determining the input coal mine image corresponding to the m-th sub-model includes: Determine the corresponding downsampling rate based on the index of the m-th sub-model; Based on the downsampling rate, the preset sample coal mine image is downsampled to obtain the input coal mine image corresponding to the m-th sub-model.

3. The method according to claim 1, characterized in that, The encoder and decoder in the same sub-model have the same number of channels; wherein, the number of channels corresponding to the m-th sub-model is determined according to the following formula: in, This represents the number of channels corresponding to the m-th sub-model.

4. The method according to any one of claims 1-3, characterized in that, The first coal mine image processing model includes: a channel expansion layer, a global encoder, a fusion layer, and a global decoder; The channel extension layer is connected to the encoder of the first sub-model in the target sub-model sequence; The global encoder is formed by sequentially concatenating the encoders of the sub-models in the target sub-model sequence; The fusion layer includes a channel dimensionality reduction layer, a fusion sub-layer, and a feature enhancement layer associated with the encoder of each sub-model; The output of the channel dimensionality reduction layer is input to the fusion sub-layer for fusion, and the fused output is input to the feature enhancement layer for enhancement. The global decoder includes at least one channel dimensionality reduction layer, and the output of the feature enhancement layer is input to the global decoder for decoding. The fusion weights of each channel dimensionality reduction layer in the fusion layer are the same.

5. The method according to claim 4, characterized in that, The step of fine-tuning the first coal mine image processing model to obtain the second coal mine image processing model further includes the following steps: The model parameters of the first coal mine image processing model are initialized, wherein the initialization includes retaining the weights of each encoder in the global encoder and configuring the weights of the global decoder based on a Gaussian distribution; The encoder in the global encoder is divided into two encoding stages according to its structural position; The weights of the first encoding stage are frozen, and the weights of the second encoding stage are fine-tuned. In response to the end of the second encoding stage fine-tuning, its weights are frozen, and the first encoding stage is fine-tuned. In response to the fulfillment of the fine-tuning termination condition, the second coal mine image processing model is obtained.

6. The method according to any one of claims 1-3, characterized in that, The decoder of the sub-model uses The single upsampling is multiplied by a factor of 1, and the upsampling method is bilinear interpolation or transposed convolution.

7. An image processing method, characterized in that, The method includes: Acquire the target coal mine image to be processed; Determine the resolution of the target coal mine image and select a second coal mine image processing model that matches the resolution; The target coal mine image is input into the second coal mine image processing model, and the prediction result of the target coal mine image is output. The second coal mine image processing model is a model obtained by using the training method described in any one of claims 1-6.

Images