Coal mine underground monitoring picture image description method

By acquiring and preprocessing video streams underground in the coal mine, extracting global and local features and generating descriptive statements, the problem of understanding massive image information is solved, monitoring efficiency and safety is improved, and safety monitoring and data analysis of coal mine sites is realized.

CN119007103APending Publication Date: 2024-11-22XIAN COAL SCI TRANSPARENT GEOLOGICAL TECH CO LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202411021747.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-29
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

Under coal mines, it is difficult for the existing technology to effectively capture and understand information in massive images, especially in complex environments. The video data is huge and complex, and the existing target recognition model cannot understand the meaning of non-obvious objects in the image, resulting in information Missed and inefficient in transmission and processing.

Method used

By obtaining the target motion clip in the video stream, preprocessing is performed to remove uneven lighting, using convolutional neural network to extract global features and object detection models to obtain anchor boxes, calculate the anchor boxes interactive properties and fuse the characteristics, and input them into self-attention language Generate the model to generate description statements, select the best description to summarize image information.

Benefits of technology

It realizes accurate identification and classification of objects, scenes and behaviors in the video, improves monitoring efficiency and security, simplifies video retrieval, and improves the safety and efficiency of coal mine sites.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119007103A_ABST
    Figure CN119007103A_ABST
Patent Text Reader

Abstract

The invention relates to a coal mine underground monitoring image description method, which is used for realizing language description under underground monitoring, preprocessing an image at the earlier stage of capturing the image, adding local object information during feature extraction to enhance the representation capability of image features, and simultaneously considering the correlation between different objects. The relevance between objects is expressed, interaction attributes are designed, and the accuracy of description generation is assisted. The method has the potential of improving safety and efficiency in the field of coal mining, safety monitoring and early warning, automatic equipment control and data analysis and decision support of a coal mine site can be achieved by applying the method, and therefore sustainable development of the coal mine industry is promoted.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a method for describing an image of a coal mine underground monitoring screen. Background Art

[0002] Image description models based on deep learning usually adopt an encoding-decoding structure. The encoder uses a convolutional neural network to extract global features from the image, while the decoder uses a recurrent neural network to generate sentence descriptions. Information is often presented in the form of text, audio, pictures, and videos. The presentation method from text to video is becoming more and more intuitive. Facing the massive visual information in the mine, how to capture and understand the effective information in the image is crucial. However, the transmission of visual information consumes a lot of energy and requires extremely high hardware. It is also time-consuming and labor-intensive to quickly obtain information from massive images. Therefore, how to realize mine image information and achieve cross-modal conversion is one of the problems that need to be solved urgently. The mine image description task captures image features and generates a complete human language to summarize the overall information of the image. It has been widely used in application scenarios such as target tracking and behavior recording, security monitoring, and illegal operations.

[0003] The mining environment is complex and there are many dangerous factors. To ensure the safety of miners and reduce the risk of accidents, real-time monitoring and data analysis of mines are required. Due to the large number of cameras inside the mine, the video data is large and complex, and each camera generates a continuous video stream. However, there is no guarantee that someone is watching every video at all times. For key behaviors that occur underground at all times, it will be very difficult and time-consuming to manually process these video data alone. At the same time, it is also difficult to retrieve and trace back historical production events, and given that the current target recognition model can only mark more obvious objects through anchor frames, it cannot understand the meaning of the image. In addition, the underground environment lacks natural light, the illumination range of fixed-position light sources is limited and uneven, and the brightness of the environment is uneven, which leads to the omission of a lot of important information. Summary of the invention

[0004] In order to overcome at least one shortcoming in the prior art, the present application provides a method for describing an image of a coal mine underground monitoring screen.

[0005] In a first aspect, a method for describing an image of a coal mine underground monitoring screen is provided, comprising:

[0006] Obtain a target motion segment in a video stream, slice and store the target motion segment, and obtain multiple images to be processed;

[0007] Preprocess each image to be processed to obtain a clear image;

[0008] Input the clear image into the convolutional neural network to obtain the global features;

[0009] Input the clear image into the target detection model to obtain multiple anchor boxes and local features representing the target location;

[0010] Calculate the interaction attributes between any two anchor boxes and concatenate all the interaction attributes to obtain additional features;

[0011] The local features and the global features are fused to obtain the fused features;

[0012] The fused features and additional features are input into the self-attention-based language generation model to generate multiple description sentences;

[0013] The best description sentence is selected from multiple description sentences as the final description sentence of the image to be processed.

[0014] In one embodiment, calculating the interaction properties between any two anchor boxes includes:

[0015]

[0016] Among them, f ij is the interaction attribute between anchor box i and anchor box j, x i is the x-axis coordinate of the diagonal intersection of anchor box i, y i is the y-axis coordinate of the diagonal intersection of anchor box i, w i is the width of anchor box i, h i is the height of anchor box i, x j is the x-axis coordinate of the diagonal intersection of anchor box j, y j is the y-axis coordinate of the diagonal intersection of anchor box j, w j is the width of anchor box j, h j is the height of anchor box j.

[0017] In one embodiment, a language generation model based on self-attention includes an encoder and a decoder; the encoder includes a first concatenation layer, an attention unit, a second concatenation layer, and an FFN; the attention unit includes multiple attention layers;

[0018] The fused features and additional features are input into the first splicing layer for feature splicing to obtain the first spliced ​​features;

[0019] The first concatenated features are input into the attention unit, and each attention layer performs a scaled dot product attention operation on the first concatenated features to obtain attention features;

[0020] All attention features are input into the second concatenation layer to obtain the second concatenated features;

[0021] The second concatenated features are input into FFN to obtain the encoded features;

[0022] The encoded features are input into the decoder to obtain multiple description sentences.

[0023] In one embodiment, the attention layer includes a first linear transformation layer, a scaled dot product attention layer, a third concatenation layer, and a second linear transformation layer;

[0024] The first concatenated features are input into the first linear transformation layer to obtain multiple sets of vectors, each set of vectors includes query content Q, keyword K and value V;

[0025] Each set of vectors is input into the scaled dot product attention layer, and the scaled dot product attention operation is performed to obtain the scaled dot product features;

[0026] All scaled dot product features are input into the third concatenation layer to obtain the third concatenated features;

[0027] The third concatenated features are input into the second linear transformation layer to obtain the attention features.

[0028] In one embodiment, the method further comprises:

[0029] The time of the corresponding image in the video stream is determined according to the final description sentence.

[0030] In one embodiment, preprocessing includes removing uneven lighting in the image to be processed.

[0031] In a second aspect, a coal mine underground monitoring image description device is provided, comprising:

[0032] An image acquisition module is used to acquire a target motion segment in a video stream, and to slice and store the target motion segment to acquire multiple images to be processed;

[0033] A preprocessing module is used to preprocess each image to be processed to obtain a clear image;

[0034] The global feature acquisition module is used to input the clear image into the convolutional neural network to obtain the global features;

[0035] An anchor frame and local feature acquisition module, used to input a clear image into a target detection model to obtain multiple anchor frames and local features representing the target position;

[0036] The additional feature acquisition module calculates the interaction attributes between any two anchor boxes and concatenates all the interaction attributes to obtain additional features;

[0037] The fusion module is used to fuse local features and global features to obtain fused features;

[0038] A description sentence generation module is used to input the fused features and additional features into the self-attention-based language generation model to generate multiple description sentences;

[0039] The best description sentence determination module is used to select the best description sentence from multiple description sentences as the final description sentence of the image to be processed.

[0040] In one embodiment, a language generation model based on self-attention includes an encoder and a decoder; the encoder includes a first concatenation layer, an attention unit, a second concatenation layer, and an FFN; the attention unit includes multiple attention layers;

[0041] The fused features and additional features are input into the first splicing layer for feature splicing to obtain the first spliced ​​features;

[0042] The first concatenated features are input into the attention unit, and each attention layer performs a scaled dot product attention operation on the first concatenated features to obtain attention features;

[0043] All attention features are input into the second concatenation layer to obtain the second concatenated features;

[0044] The second concatenated features are input into FFN to obtain the encoded features;

[0045] The encoded features are input into the decoder to obtain multiple description sentences.

[0046] In a third aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned coal mine underground monitoring image description method is implemented.

[0047] In a fourth aspect, a computer program product is provided, including a computer program / instruction, which, when executed by a processor, implements the above-mentioned coal mine underground monitoring image description method.

[0048] Compared with the prior art, this application has the following beneficial effects:

[0049] 1. By identifying and classifying objects, scenes, and behaviors in the video, we can more accurately understand the information contained in the video and extract valuable content from it. For example, in surveillance cameras, video content description can help quickly locate and identify suspicious behaviors, thereby improving monitoring efficiency and security.

[0050] 2. Slice and label video data so that users can easily find and retrieve specific content. By segmenting the video and generating a slice index, users can quickly jump to a specified video segment without having to browse the entire video, which helps improve video retrieval efficiency and user experience.

[0051] 3. It has the potential to improve safety and efficiency in the field of coal mining. By applying this method, safety monitoring and early warning, automated equipment control, data analysis and decision support can be achieved on-site in coal mines, thereby promoting the sustainable development of the coal mining industry. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The present application may be better understood by referring to the following description given in conjunction with the accompanying drawings, which together with the following detailed description are included in this specification and form a part of this specification. In the drawings:

[0053] Figure 1 A flowchart of a method for describing underground coal mine monitoring images according to an embodiment of the present application is shown;

[0054] Figure 2 A schematic diagram of a language generation model based on self-attention according to an embodiment of the present application is shown;

[0055] Figure 3 A schematic diagram of the structure of the attention layer is shown;

[0056] Figure 4 A structural block diagram of a coal mine underground monitoring image description device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0057] The exemplary embodiments of the present application will be described below in conjunction with the accompanying drawings. For the sake of clarity and conciseness, not all features of the actual embodiments are described in the specification. However, it should be understood that many implementation-specific decisions can be made in the process of developing any such actual embodiments in order to achieve the specific goals of the developer, and these decisions may vary from embodiment to embodiment.

[0058] It is also necessary to explain here that, in order to avoid obscuring the present application due to unnecessary details, only the device structure closely related to the scheme according to the present application is shown in the drawings, while other details that are not closely related to the present application are omitted.

[0059] It should be understood that the present application is not limited to the described implementation forms due to the following description with reference to the accompanying drawings. In this article, where feasible, the embodiments can be combined with each other, features between different embodiments can be replaced or borrowed, and one or more features can be omitted in one embodiment.

[0060] The present application provides a method for describing underground monitoring images in coal mines. For images in mines, the environment is dim, the light source is complex and the brightness is inconsistent, there is a lot of dust in the mine, and the suspended particles will absorb the reflected light source, which will reduce the clarity and visibility of the image. In addition, the expression ability of a single feature is limited and it is impossible to capture rich contextual information. Therefore, before the image is captured, the brightness of the image is adjusted and the illumination is filtered. Local object information is added during feature extraction to enhance the representation ability of image features. At the same time, considering the correlation between different objects and the correlation between objects, interactive attributes are designed to assist in generating the accuracy of the description. Finally, the described information is stored through natural language processing for retrieval, including the time of description, description content, camera ID, camera location, etc.

[0061] Figure 1 A flowchart of a method for describing underground coal mine monitoring images according to an embodiment of the present application is shown. Figure 1 , a coal mine underground monitoring image description method, comprising:

[0062] Step S1, obtaining a target motion segment in a video stream, and slicing and storing the target motion segment to obtain a plurality of images to be processed.

[0063] The video stream is detected frame by frame to detect the target's motion state and record the start and end time of the target's motion state. Here, the target may be, for example, an underground working device in a coal mine, and the video stream captured during the start and end time is the target motion segment.

[0064] Step S2, preprocessing each image to be processed to obtain a clear image.

[0065] In order to make the target image clearer, this step aims to reduce the halo of the ambient light source and eliminate the fogging caused by dust. It can not only reduce the impact of local light effects, but also reduce halo artifacts, thereby improving the accuracy of the underground environment description.

[0066] Specifically, the preprocessing includes removing uneven illumination in the image to be processed.

[0067] Here, for each pixel in the image to be processed, the maximum value in the three color channels of R, G, and B can be removed to remove uneven illumination in the image. In addition, dust reflection in the image can be removed. After the above preprocessing, a clear image can be obtained.

[0068] Step S3: Input the clear image into a convolutional neural network to obtain global features. Here, the convolutional neural network may be, for example, ResNet101 or other convolutional neural networks, which are not specifically limited here.

[0069] Step S4, inputting the clear image into the target detection model to obtain multiple anchor frames and local features representing the target position. Here, the target detection model adopts an existing target detection model, such as Faster R-CNN, which captures the target in the image through the target recognition model and identifies the target with an anchor frame.

[0070] Step S5, calculate the interaction attributes between any two anchor boxes, and concatenate all the interaction attributes to obtain additional features.

[0071] Here, the target object interaction properties are designed in an attempt to enhance the interactivity and correlation between objects in the image, thereby further improving the description accuracy.

[0072] Step S6: local features and global features are fused to obtain fused features.

[0073] Step S7: The fused features and additional features are input into a self-attention-based language generation model to generate multiple description sentences.

[0074] Step S8, selecting the best description sentence from multiple description sentences as the final description sentence of the image to be processed.

[0075] Here, evaluation indicators such as BLEU (Bilingual Evaluation Understudy) and CIDEr (Consensus-based Image Description Evaluation) are used. CIDEr independently generates descriptions based on consistency evaluation to select the best description statement as the final description statement of the image to be processed. The final description statement can provide detailed mine site conditions for managers to quickly understand the mine status.

[0076] In one embodiment, in step S5, the interactive attribute between any two anchor boxes is calculated and the attribute is used as an additional feature to assist in generating image description, including:

[0077]

[0078] Among them, f ij is the interaction attribute between anchor box i and anchor box j, x i is the x-axis coordinate of the diagonal intersection of anchor box i, y i is the y-axis coordinate of the diagonal intersection of anchor box i, w i is the width of anchor box i, h i is the height of anchor box i, x j is the x-axis coordinate of the diagonal intersection of anchor box j, y j is the y-axis coordinate of the diagonal intersection of anchor box j, wj is the width of anchor box j, h j is the height of anchor box j.

[0079]

[0080] w i =(x i_max -x i_min )+1

[0081] h i =(y i_max -y i_min )+1

[0082] Among them, x i_max 、x i_min ,y i_max ,y i_min are the four vertices of the anchor box, x i_max 、x i_min are the vertices with larger x-coordinate values ​​and the vertices with smaller x-coordinate values ​​along the x-axis of anchor box i, respectively. i_max ,y i_min They are the vertices with larger y coordinate values ​​and smaller x coordinate values ​​of anchor box i along the y-axis direction. Here, the x-axis direction is the width direction and the y-axis direction is the height direction. Anchor box i adopts the quaternion (x i ,y i , w i ,h i )express.

[0083] In one embodiment, Figure 2 A schematic diagram of a language generation model based on self-attention according to an embodiment of the present application is shown, see Figure 2 The language generation model based on self-attention includes an encoder and a decoder. Here, the decoder adopts the decoder in the existing language generation model, such as the decoder in Transformer.

[0084] The encoder includes a first concatenation layer, an attention unit, a second concatenation layer, and a FFN (Feed Forward Networks); the attention unit includes multiple attention layers;

[0085] The fused feature Vm and the additional feature are input into the first splicing layer for feature splicing to obtain the first spliced ​​feature;

[0086] The first concatenated features are input into the attention unit, and each attention layer performs a scaled dot product attention operation on the first concatenated features to obtain attention features;

[0087] All attention features are input into the second concatenation layer to obtain the second concatenated features;

[0088] The second concatenated features are input into FFN to obtain the encoded features;

[0089] The encoded features are input into the decoder to obtain multiple description sentences.

[0090] In this embodiment, a language generation model based on self-attention is constructed to solve the problem that the traditional language generation models RNN and LSTM will have gradient disappearance as the complexity of the sentence increases, resulting in the lack of relevance between the sentences. The language generation model includes a variety of input data, including both the integration of image features and interactive information, making the final generated description language more accurate and reliable.

[0091] Figure 3 shows a schematic diagram of the structure of the attention layer, see Figure 3 , there are 6 attention layers designed, each of which includes the first linear transformation layer (Linear), the scaled dot product attention layer (Scaled Dot-Product Attention), the third concatenation layer (Concat) and the second linear transformation layer (Linear);

[0092] The first concatenated features are input into the first linear transformation layer to obtain multiple sets of vectors, each set of vectors includes query content Q, keyword K and value V; the design of multiple sets of vectors allows the model to focus on different information in different representation subspaces in parallel, thereby enhancing the expressiveness and robustness of the model.

[0093] Each set of vectors is input into the scaled dot product attention layer, and the scaled dot product attention operation is performed to obtain the scaled dot product features;

[0094] All scaled dot product features are input into the third concatenation layer to obtain the third concatenated features;

[0095] The third concatenated features are input into the second linear transformation layer to obtain the attention features.

[0096] Furthermore, the method further comprises: determining the time of the corresponding image in the video stream according to the final description sentence. Here, the corresponding image can be determined according to the final description sentence, and the time of the image in the video stream can be accurately located, so as to retrieve the historical video stream.

[0097] Based on the same inventive concept as the coal mine underground monitoring image description method, this embodiment also provides a corresponding coal mine underground monitoring image description device. Figure 4 The structure block diagram of the coal mine underground monitoring image description device according to an embodiment of the present application is shown, including:

[0098] The image acquisition module 41 is used to acquire a target motion segment in a video stream, and to slice and store the target motion segment to acquire a plurality of images to be processed;

[0099] A preprocessing module 42 is used to preprocess each image to be processed to obtain a clear image;

[0100] A global feature acquisition module 43 is used to input the clear image into the convolutional neural network to obtain global features;

[0101] An anchor frame and local feature acquisition module 44, used for inputting the clear image into the target detection model to obtain multiple anchor frames and local features representing the target position;

[0102] The additional feature acquisition module 45 calculates the interaction attributes between any two anchor frames and concatenates all the interaction attributes to obtain additional features;

[0103] A fusion module 46 is used to fuse local features and global features to obtain fused features;

[0104] A description sentence generation module 47, used for inputting the fused features and the additional features into a self-attention-based language generation model to generate multiple description sentences;

[0105] The best description sentence determination module 48 is used to select the best description sentence from multiple description sentences as the final description sentence of the image to be processed.

[0106] The coal mine underground monitoring image description device of this embodiment has the same inventive concept as the coal mine underground monitoring image description method mentioned above, so the specific implementation method of the device can be seen in the implementation example part of the coal mine underground monitoring image description method in the above text, and its technical effect corresponds to the technical effect of the above method, which will not be repeated here.

[0107] An embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned coal mine underground monitoring image description method is implemented.

[0108] An embodiment of the present application provides a computer program product, including a computer program / instruction. When the computer program / instruction is executed by a processor, the above-mentioned coal mine underground monitoring image description method is implemented.

[0109] In summary, this application has the following technical effects:

[0110] 1. By identifying and classifying objects, scenes, and behaviors in the video, we can more accurately understand the information contained in the video and extract valuable content from it. For example, in surveillance cameras, video content understanding technology can help quickly locate and identify suspicious behaviors, thereby improving monitoring efficiency and security.

[0111] 2. Slice and label video data so that users can easily find and retrieve specific content. By segmenting the video and generating a slice index, users can quickly jump to a specified video segment without having to browse the entire video, which helps improve video retrieval efficiency and user experience.

[0112] 3. It has the potential to improve safety and efficiency in the field of coal mining. By applying this technology, safety monitoring and early warning, automated equipment control, data analysis and decision support can be achieved on-site in coal mines, thereby promoting the sustainable development of the coal mining industry.

[0113] The above are only various implementations of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for describing an image of a coal mine underground monitoring screen, characterized in that: include: Obtain a target motion segment in a video stream, and slice and store the target motion segment to obtain a plurality of images to be processed; Preprocessing each of the to-be-processed images to obtain a clear image; Inputting the clear image into a convolutional neural network to obtain global features; Inputting the clear image into a target detection model to obtain a plurality of anchor frames and local features representing target locations; Calculate the interaction attributes between any two anchor boxes and concatenate all the interaction attributes to obtain additional features; The local feature and the global feature are fused to obtain a fused feature; The fused features and the additional features are input into a self-attention-based language generation model to generate multiple description sentences; An optimal description sentence is selected from the multiple description sentences as a final description sentence of the image to be processed.

2. The method according to claim 1, characterized in that in, Calculate the interaction properties between any two anchor boxes, including: Among them, f ij is the interaction attribute between anchor box i and anchor box j, x i is the x-axis coordinate of the diagonal intersection of anchor box i, y i is the y-axis coordinate of the diagonal intersection of anchor box i, w i is the width of anchor box i, h i is the height of anchor box i, x j is the x-axis coordinate of the diagonal intersection of anchor box j, y j is the y-axis coordinate of the diagonal intersection of anchor box j, w j is the width of anchor box j, h j is the height of anchor box j.

3. The method according to claim 1, characterized in that The self-attention-based language generation model includes an encoder and a decoder; the encoder includes a first splicing layer, an attention unit, a second splicing layer, and FFN; the attention unit includes multiple attention layers; The fused features and the additional features are input into the first splicing layer for feature splicing to obtain first spliced ​​features; The first concatenated features are input into the attention unit, and each of the attention layers performs a scaled dot product attention operation on the first concatenated features to obtain an attention feature; All attention features are input into the second concatenated layer to obtain the second concatenated features; The second concatenated features are input into the FFN to obtain encoded features; The encoded features are input into the decoder to obtain a plurality of description sentences.

4. The method according to claim 3, characterized in that The attention layer includes a first linear transformation layer, a scaled dot product attention layer, a third concatenation layer, and a second linear transformation layer; The first concatenated features are input into the first linear transformation layer to obtain multiple groups of vectors, each group of vectors includes query content Q, keyword K and value V; Each set of vectors is input into the scaled dot product attention layer, and the scaled dot product attention operation is performed to obtain the scaled dot product features; All scaled dot product features are input into the third concatenated layer to obtain the third concatenated features; The third concatenated feature is input into the second linear transformation layer to obtain the attention feature.

5. The method according to claim 1, characterized in that The method further comprises: The time of the corresponding image in the video stream is determined according to the final description sentence.

6. The method according to claim 1, characterized in that The preprocessing includes removing uneven illumination in the image to be processed.

7. A coal mine underground monitoring image description device, characterized in that: include: An image acquisition module is used to acquire a target motion segment in a video stream, and to slice and store the target motion segment to acquire a plurality of images to be processed; A preprocessing module, used for preprocessing each of the images to be processed to obtain a clear image; A global feature acquisition module, used for inputting the clear image into a convolutional neural network to obtain global features; An anchor frame and local feature acquisition module, used for inputting the clear image into a target detection model to obtain multiple anchor frames and local features representing target positions; The additional feature acquisition module calculates the interaction attributes between any two anchor boxes and concatenates all the interaction attributes to obtain additional features; A fusion module, used for fusing the local features with the global features to obtain fused features; A description sentence generation module, used for inputting the fused features and the additional features into a self-attention-based language generation model to generate multiple description sentences; The best description sentence determination module is used to select the best description sentence from the multiple description sentences as the final description sentence of the image to be processed.

8. The device according to claim 7, characterized in that The self-attention-based language generation model includes an encoder and a decoder; the encoder includes a first splicing layer, an attention unit, a second splicing layer, and FFN; the attention unit includes multiple attention layers; The fused features and the additional features are input into the first splicing layer for feature splicing to obtain first spliced ​​features; The first concatenated features are input into the attention unit, and each of the attention layers performs a scaled dot product attention operation on the first concatenated features to obtain an attention feature; All attention features are input into the second concatenated layer to obtain the second concatenated features; The second concatenated features are input into the FFN to obtain encoded features; The encoded features are input into the decoder to obtain a plurality of description sentences.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the coal mine underground monitoring image description method according to any one of claims 1-6.

10. A computer program product, characterized in that It includes a computer program / instruction, which, when executed by a processor, implements the coal mine underground monitoring image description method described in any one of claims 1-6.

Citation Information

Patent Citations

  • Multi-feature fusion and time-space attention mechanism combination-based video description method

    CN108388900A

  • Image processing method, equipment, computer storage medium and server

    CN108665506A

  • Differential description statement generation method and device, equipment and medium

    CN114511860A

  • Multiband image description generation method based on feature fusion

    CN116645666A