Mining image sensor system based on fusion RNN

Abstract

The invention discloses a mining image sensor system based on fusion RNN, and relates to the technical field of image processing and computer vision, the mining image sensor system comprises an image acquisition module, a preprocessing module, a convolutional neural network module and a post-processing module; the image acquisition module is used for acquiring an original mine image; the preprocessing module is used for acquiring a preprocessed mine image by utilizing an image processing library according to the original mine image; the convolutional neural network module is used for obtaining a classification detection result by using a deep learning model based on the preprocessed mine image; and the post-processing module is used for performing post-processing on the classification detection result by using the image processing library to obtain a post-processed mine image. According to the invention, by combining the CNN and the image database, efficient and accurate image processing and analysis are realized.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing and computer vision, and in particular to a mining image sensor system based on fused RNN. Background Art

[0002] Existing image sensor technology has certain limitations in image acquisition, processing and analysis. Traditional image processing methods rely on feature extraction algorithms manually designed through Opencv algorithms, and then directly hand over the processing results to convolutional neural networks for target confirmation. Although this process can build a sensor system, this development model does not support the deployment of expansion devices such as relays. When expanding multiple applications, the framework can hardly support it and cannot call hardware resources relatively efficiently.

[0003] In traditional mining image sensor systems, the equipment is usually single and lacks scalability and modular design. Therefore, in a multi-device and multi-task environment, the system finds it difficult to support relay functions and cannot efficiently schedule and manage resources, resulting in performance bottlenecks and increased system complexity.

[0004] Therefore, there is an urgent need for a mining image sensor system based on fusion RNN to improve the efficiency and accuracy of on-site image recognition of mining image sensors in the complex working environment of mines. Summary of the invention

[0005] The purpose of the present invention is to propose a mining image sensor system based on fusion RNN, which adopts the basic method of factory mode and realizes efficient and accurate image processing and analysis of the mining image sensor system by fusing convolutional neural network and image processing library.

[0006] To achieve the above object, the present invention provides a mining image sensor system based on fusion RNN, the mining image sensor system includes an image acquisition module, a preprocessing module, a convolutional neural network module and a post-processing module;

[0007] The image acquisition module is used to acquire original mine images;

[0008] The preprocessing module is used to obtain a preprocessed mine image using an image processing library according to the original mine image;

[0009] The convolutional neural network module is used to obtain classification detection results based on the pre-processed mine image using a deep learning model;

[0010] The post-processing module is used to use the image processing library to post-process the classification detection result to obtain a mine image;

[0011] Among them, the deep learning model includes a convolutional neural network and a feature extraction network.

[0012] Optionally, obtaining a pre-processed mine image using an image processing library according to the original mine image includes:

[0013] The image database is used to sequentially perform grayscale conversion, denoising and edge detection on the original mine image to obtain the pre-processed mine image.

[0014] Optionally, obtaining a classification detection result based on the preprocessed mine image using a deep learning model includes:

[0015] Inputting the preprocessed mine image into a convolutional neural network to obtain an original mine image output by the convolutional neural network;

[0016] Using the feature extraction network to extract features from the original mine image output by the convolutional neural network to obtain pre-processed mine graphic features;

[0017] Classification and detection are performed in sequence according to the pre-processed mine image features to obtain the classification detection result;

[0018] Wherein, the convolutional neural network includes a convolutional layer, a pooling layer and a fully connected layer.

[0019] Optionally, obtain a convolutional neural network including:

[0020] Collect actual mine image data;

[0021] Annotating the target object category, position boundary and boundary box according to the actual mine image data to obtain annotated actual mine image data;

[0022] According to the annotated actual mine image data, the image size is adjusted, the pixel value is normalized, and the data is enhanced in sequence to obtain the pre-processed actual mine image as training data;

[0023] The convolutional neural network is obtained by training using a back propagation algorithm according to the training data.

[0024] Optionally, the mining image sensor system further includes a distributed analyzer module;

[0025] The distributed analyzer module is used for supporting dynamic loading and unloading of the mining image sensor system based on a distributed analyzer in a factory mode.

[0026] Optionally, the mining image sensor system further includes a multi-model fusion module;

[0027] The multi-model fusion module is used to load and fuse the deep learning networks in sequence by using decision layer fusion to obtain a fused RNN model.

[0028] Optionally, the decision layer is fused as:

[0029]

[0030] Among them, Final Decision is the final decision result, arg max(·) represents the maximum value operation function, n represents the total number of candidate items, C i is a candidate, C is a decision candidate, δ(C i ,C) represents the function δ used to measure candidate C i The similarity or consistency with the current candidate C.

[0031] Optionally, the mining image sensor system further includes an initialization calculation management module, a scheduling task management module, a feature recognition optimization module and a model update module;

[0032] The initialization calculation management module is used to scan and register the device and its hardware resources to obtain the hardware status and task requirements of the device;

[0033] The scheduling task management module is used to obtain NPU calculation results using the neural network processor according to the hardware status of the device and the task requirements;

[0034] The feature recognition optimization module is used to perform feature recognition, weight adjustment and update on the deep learning model in sequence according to the NPU calculation result to obtain a feature recognition optimization result;

[0035] The model updating module is used to realize continuous optimization and updating of the deep learning model by using online learning and meta-learning technology according to the feature recognition optimization result.

[0036] Optionally, the initialization calculation management module includes a system initialization unit and an algorithm initialization unit;

[0037] The system initialization unit is used to scan and register the device to obtain the hardware status of the device;

[0038] The algorithm initialization unit is used to scan and register the hardware resources of the device to obtain the task requirements of the device.

[0039] Optionally, the mining image sensor system further includes a cross-device communication module and a safety fault-tolerant module;

[0040] The cross-device communication module is used for data transmission and communication across devices;

[0041] The safety fault-tolerant module is used to provide a fault detection and recovery mechanism for the data transmission and communication.

[0042] The present invention has the following beneficial effects:

[0043] The present invention achieves efficient and accurate image processing and analysis by integrating convolutional neural networks and image processing libraries. Compared with traditional methods, the present invention has the following advantages:

[0044] The present invention realizes automatic feature extraction by using convolutional neural network (CNN) instead of traditional manual feature design. CNN can automatically learn and extract image features through a large amount of training data, and randomly selects targets with relatively high confidence by adding a timing loop, and throws them into incremental update processing, thereby avoiding the complexity and limitations of manually designed feature update models; CNN trained with a large amount of data has a high accuracy rate in image classification and target detection tasks; OpenCV provides rich image processing functions, which is convenient for various image pre-processing and post-processing operations; the software factory mode enables the system to have good scalability and modular design, which is convenient for subsequent function expansion and optimization; the present invention adopts a distributed analyzer system architecture based on the factory mode, supports dynamic loading and unloading of sensor devices, realizes the parallel execution of multiple image processing tasks, and allows multiple sensor devices and analysis nodes to work together in the same network environment, and performs data processing through edge computing nodes, supporting the deployment of relay functions. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0046] Figure 1 This is a flow chart of a mining image sensor system based on RNN fusion according to an embodiment of the present invention. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solutions and advantages of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in combination with the specific embodiments of the present invention and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] The terms used in the embodiments of the present invention are only used to explain the specific embodiments of the present invention and are not intended to limit the present invention.

[0049] In view of the shortcomings of the prior art, the present invention adopts a distributed analyzer architecture based on the factory mode to support dynamic loading and unloading of sensor devices. The specific improvements are as follows:

[0050] Distributed Analyzer Architecture: The system distributes the analysis tasks across multiple nodes, each of which can independently perform specific image processing tasks. Through this architecture, the system can support multiple relay nodes for distributed processing and transmission of data.

[0051] Dynamic loading and unloading: The system supports dynamic loading and unloading of analyzers, and can adjust the deployment of analyzers in real time according to task requirements. This dynamism makes the system highly scalable and flexible in a multi-device environment.

[0052] High-speed 10G ring network: Connecting various analysis nodes through a 10G ring network enables high-speed data transmission and real-time processing, supports efficient deployment of relay functions, and this architecture can provide stable and efficient data processing and transmission services in complex mine environments.

[0053] like Figure 1 As shown, an embodiment of the present invention provides a mining image sensor system based on fusion RNN, and the mining image sensor system includes an image acquisition module, a preprocessing module, a convolutional neural network module and a post-processing module;

[0054] An image acquisition module, used for acquiring original mine images;

[0055] A preprocessing module is used to perform preprocessing operations on the original mine image using an image processing library (OpenCV) to obtain a preprocessed mine image. The preprocessing process includes grayscale conversion, denoising, edge detection and other operations, which improves the image quality;

[0056] The convolutional neural network module is used to input the preprocessed mine image into the deep learning model, extract high-level image features through the multi-layer convolution, pooling and full-level and feature extraction network of the convolutional neural network, and perform classification and detection according to task requirements to obtain classification and detection results. Specifically:

[0057] After the deep learning model receives the pre-processed mine image, the system will throw it to the scheduling task management module according to the configured parameter requirements. The scheduling task management module schedules different calculations according to the configured calculation attributes and then the input image attributes. Due to the complex environment of the mine, the processing tasks under different time sequences are different. For example, when the belt is running, high-speed and high-computing power are required to detect the belt status. When the belt stops, high-precision personnel safety needs to be detected. Different multi-layer convolution and pooling strategies are required. In this way, for input images of different sizes, the system adopts an adaptive pooling strategy to ensure that the dimensions of the output features are consistent. This method can improve the generalization ability of the model and adapt to the input of a variety of different scenarios. In the image processing process, the system combines edge detection features, texture features, and shape features to improve the accuracy and robustness of target recognition.

[0058] Among them, the convolutional neural network is trained using training data, specifically:

[0059] Source and preprocessing of training data:

[0060] Data source: The training data mainly comes from actual image collection in the mine environment. By deploying sensors at different locations, actual mine image data with different lighting, angles and scenes is obtained to ensure the diversity and representativeness of the data.

[0061] Data labeling: Semi-automatic labeling tools are used in combination with manual verification to ensure the accuracy and consistency of data labeling. The labeling content includes information such as the target object category, location, and bounding box in the image.

[0062] Data preprocessing: Before training, the data is standardized, including adjusting the image size, normalizing pixel values, and data augmentation (such as rotation, flipping, cropping, etc.) to improve the robustness and generalization ability of the model.

[0063] Training methods and hyperparameter tuning:

[0064] Training algorithm: Use the back propagation algorithm to train the model using the training data, and optimize the model parameters by minimizing the cross entropy loss function;

[0065] Hyperparameter tuning: During the training process, hyperparameters such as learning rate, regularization parameter, batch size, etc. are tuned through grid search or random search to ensure the best performance of the model on the validation set;

[0066] Iteration strategy: The strategy of gradually reducing the learning rate is used for training. The initial learning rate is set to 0.01 and is reduced by half every 10 epochs until the optimal point is reached. The early stopping strategy is also used during training to prevent overfitting.

[0067] The post-processing module is used to post-process the classification detection results in combination with OpenCV to generate intuitive output results as mine images. The post-processing includes drawing bounding boxes, labels, etc. for easy display and understanding.

[0068] In order to improve development efficiency, the present invention adopts the software factory model, modularizes the image processing and analysis functions, forms a standardized component library, and each component is called through an interface to support flexible system configuration and function expansion.

[0069] The mining image sensor system connects the edge computing nodes arranged at multiple levels through a high-speed 10G ring network to achieve dynamic loading, model matching and multi-model fusion analysis of multiple image sensor data. The system uses convolutional neural networks for target confirmation and realizes parallel execution and efficient scheduling of multiple image processing tasks through an adaptive hardware resource allocation mechanism. In addition, the system supports dynamic loading and unloading of multiple models, and realizes dynamic management and resource allocation of models through the factory mode, which greatly improves the flexibility and scalability of the system; using the edge computing nodes arranged at multiple levels, the data processing and analysis functions are integrated into the data source, reducing data transmission delays and broadband occupancy, and improving the response speed and real-time performance of the system.

[0070] The mining image sensor system also includes a distributed analyzer module, which is used for distributed analyzers based on the factory mode, supports dynamic loading and unloading of the mining image sensor system, and realizes the parallel execution of multiple image processing tasks. Each factory mode distributed analyzer node is responsible for a specific task, and realizes task collaboration and data exchange through network connection. This design allows multiple mining image sensor systems and analysis nodes to work together in the same network environment, and perform data processing through edge computing nodes, supporting the deployment of relay functions. In addition, the mining image sensor system can be connected through a 10G ring network to achieve high-speed data transmission and real-time processing, further enhancing the scalability and flexibility of the system.

[0071] The mining image sensor system also includes a multi-model fusion module, which is used to support the simultaneous loading and fusion of multiple deep learning models, including convolutional neural networks and feature extraction algorithms. These models work together in different image processing tasks to improve the accuracy of target recognition and analysis. Specifically:

[0072] The multi-model fusion module also involves the scheduling task management module. For example, the traditional video surveillance system in the current mine field usually relies only on video data and lacks effective use of audio signals, resulting in incomplete and inaccurate identification of potential dangers. In addition, most current systems usually use a single deep learning model when processing multimodal (audio and video) data, which has problems such as high computing resource consumption, slow response speed, and limited detection accuracy. Here, multiple deep learning models are dynamically loaded and managed through the engineering mode. The system can adjust the operating status of the model in real time (such as start, pause, or terminate) according to the current computing resources and task requirements to ensure efficient processing and analysis of audio and video data. At the same time, a model fusion strategy is adopted to comprehensively judge the detection results of different models to improve the accuracy and response speed of recognition.

[0073] The multi-model fusion module adopts decision-level fusion (or hybrid fusion), specifically:

[0074] Each image or audio input will get an inference result. The result is regarded as a voting process, and the result with the most occurrences is selected as the final output. Assuming there are 3 models, anomalies occur in both video and audio recognition. The system counts each result for calculation:

[0075]

[0076] Among them, Final Decision is the final decision result, which is obtained by selecting the candidate C that maximizes a certain evaluation function; arg max(·) represents the maximum value operation function, where C is the decision candidate; For each candidate C i Perform some measurement or similarity calculation with the current candidate C, and sum all the measurement results, where n represents the total number of candidates; δ(C i ,C) represents the function δ used to measure candidate C i The similarity or consistency with the current candidate C. This measurement function can be defined in different forms according to the specific problem, such as voting, probability, similarity, etc. This formula represents the similarity or consistency with the current candidate C from the candidate set {C1, C2, …, C n} select a C such that C is consistent with all other candidate Cs i In other words, the final decision is made based on the overall consistency with all other candidates.

[0077] In addition, there may be a situation where only the audio but the video does not change. At this time, the maximum confidence method is used. The confidence of each model's output result is used to select the output with the highest confidence as the final decision. Simply compare the confidence values ​​of all models and select the highest one as the final output.

[0078] The mining image sensor system also includes an initialization calculation management module, a scheduling task management module, a feature recognition optimization module and a model update module;

[0079] Initialize the computing management module, scan and register all available devices and their hardware resources to obtain task requirements and device status;

[0080] The initialization computing management module is mainly responsible for managing and monitoring all device resources within the entire AI system service framework, including the status of hardware resources such as CPU, GPU, NPU, etc., to achieve cross-device hardware resource scheduling, and dynamically bind the hardware resources of one or more devices to meet computing task requirements. At this time, the device will synchronize part of the device information, prioritize computing resources, and use distributed computing algorithms to decompose and schedule tasks to ensure that computing tasks can be efficiently allocated to each device.

[0081] The initialization calculation management module includes a system initialization unit and an algorithm initialization unit;

[0082] The system initialization unit is used to scan and register all available devices to obtain the hardware status of the devices:

[0083] Function: System initialization involves configuring and preparing various parts of the entire system to ensure that the system can operate normally and communicate with the external environment, which includes but is not limited to initializing hardware devices, establishing communication links, loading drivers, setting basic parameters, etc.

[0084] Typical steps: usually include power-on self-test (POST), starting the boot loader (Bootloader), initializing the operating system and its services, etc.

[0085] The algorithm initialization unit is used to scan and register the hardware resources of the device to obtain the task requirements of the device:

[0086] Function: Algorithm initialization focuses on preparing specific algorithms or models in the system so that they can run and process data correctly in subsequent tasks.

[0087] Typical steps: The algorithm initialization process may include loading model parameters, configuring algorithm hyperparameters, preparing the processing flow of input data, etc.

[0088] The algorithm initialization unit usually relies on the basic environment after the system initialization unit is completed. For example, the algorithm may need to access initialized hardware devices or system services. Usually, the system initialization unit is executed before the algorithm initialization unit to ensure the stability and availability of the system. In some cases, the algorithm initialization unit may be delayed until the system has reached a certain state or condition to ensure the smooth progress of the initialization process and the correct configuration of the algorithm.

[0089] The scheduling task management module is used to use the neural network processor to obtain the NPU calculation results according to the hardware status of the device and the task requirements. That is, according to the task requirements and device status, the task scheduling and allocation module is called to perform task decomposition and resource allocation. Specifically:

[0090] This module performs task allocation and scheduling based on the hardware status of the device and the requirements of the computing task. The core hardware scheduling unit is the neural network processor (NPU) as the main computing unit to process multi-level 8-bit integer operations, which is suitable for tasks such as AI reasoning and model updates, and dynamically allocates the spare computing power of other devices (such as CPU, GPU) to assist in feature recognition and model optimization. In addition, when each device performs the assigned computing tasks, the NPU processes the main AI reasoning tasks, and other devices perform auxiliary calculations and feature extraction.

[0091] The feature recognition module optimization module is used to perform feature recognition, weight adjustment and update of the deep learning model in turn according to the NPU calculation results to obtain feature recognition optimization results, that is, automatically identify features according to the NPU calculation results, adjust model weights, and achieve self-update through online learning. Specifically:

[0092] Responsible for automated feature recognition, extracting features from input data based on AI algorithms, improving the model's prediction accuracy and computing efficiency by optimizing computing resources and feature extraction algorithms, and using distributed computing capabilities to adjust the model's weights and parameters in real time to facilitate self-update of the model.

[0093] The model updating module is used to realize the continuous optimization and updating of the deep learning model by using online learning and meta-learning technology according to the feature recognition optimization results. Specifically:

[0094] This module monitors the performance of the model in real time and automatically adjusts the model parameters based on new data and feature recognition results. It uses online learning and meta-learning techniques to achieve continuous optimization and self-update of the model. It provides model update logs and performance monitoring reports for system administrators to refer to and adjust.

[0095] The mining image sensor system also includes a cross-device communication module and a safety fault-tolerant module;

[0096] The cross-device communication module is used for cross-device data transmission and communication to ensure the real-time and accuracy of task data transmission. Specifically:

[0097] This module supports high-speed communication between devices, ensuring efficient data transmission and instruction execution of computing tasks. It reduces data transmission delays and improves overall system performance through optimized network protocols and communication strategies.

[0098] The safety fault-tolerant module is used to provide fault detection and recovery mechanisms for data transmission and communication in cross-device communication to ensure safe and stable operation of the system. Specifically:

[0099] Provide data security protection, ensure data integrity and confidentiality in cross-device communications, implement fault detection and recovery mechanisms for computing tasks, and ensure high availability and reliability of the system.

[0100] The distributed analyzer system based on the factory mode is particularly suitable for image analysis and processing in the mine environment. The distributed analyzer system based on the factory mode can support cross-device calls to the spare hardware resources of other devices and bind the hardware of one or more devices to optimize resource utilization and computing performance. After obtaining a specific computing task, the system will intelligently allocate operations, and each module will perform corresponding tasks. The core hardware scheduling unit is the neural network processor (NPU), which mainly processes multi-level 8-bit integer operations. At the same time, the system uses other schedulable computing power to automatically identify features, optimize overall weights, update models, and complete the function of self-model update.

[0101] The mine sensor system of the present invention supports connecting multi-level edge computing nodes through a high-speed 10G ring network to achieve dynamic loading, model matching and multi-model fusion analysis of multiple image sensor data. The system uses convolutional neural networks for target confirmation, and through an adaptive hardware resource allocation mechanism, it achieves parallel execution and efficient scheduling of multiple image processing tasks. In addition, the system supports dynamic loading and unloading of multiple models, and realizes dynamic management and resource allocation of models through the factory mode, which greatly improves the flexibility and scalability of the system.

[0102] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0103] The present invention is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0104] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0105] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0106] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A mining image sensor system based on fusion RNN, characterized in that: The mining image sensor system includes an image acquisition module, a preprocessing module, a convolutional neural network module and a post-processing module; The image acquisition module is used to acquire original mine images; The preprocessing module is used to obtain a preprocessed mine image using an image processing library according to the original mine image; The convolutional neural network module is used to obtain classification detection results based on the pre-processed mine image using a deep learning model; The post-processing module is used to use the image processing library to post-process the classification detection result to obtain a mine image; Among them, the deep learning model includes a convolutional neural network and a feature extraction network.

2. The mining image sensor system based on fusion RNN according to claim 1 is characterized in that: Acquiring a pre-processed mine image using an image processing library according to the original mine image includes: The image database is used to sequentially perform grayscale conversion, denoising and edge detection on the original mine image to obtain the pre-processed mine image.

3. The mining image sensor system based on fusion RNN according to claim 2 is characterized in that: Obtaining classification detection results based on the preprocessed mine image using a deep learning model includes: Inputting the preprocessed mine image into a convolutional neural network to obtain an original mine image output by the convolutional neural network; Using the feature extraction network to extract features from the original mine image output by the convolutional neural network to obtain pre-processed mine graphic features; Classification and detection are performed in sequence according to the pre-processed mine image features to obtain the classification detection result; Wherein, the convolutional neural network includes a convolutional layer, a pooling layer and a fully connected layer.

4. The mining image sensor system based on fusion RNN according to claim 3 is characterized in that: Obtaining a convolutional neural network involves: Collect actual mine image data; Annotating the target object category, position boundary and boundary box according to the actual mine image data to obtain annotated actual mine image data; According to the annotated actual mine image data, the image size is adjusted, the pixel value is normalized, and the data is enhanced in sequence to obtain the pre-processed actual mine image as training data; The convolutional neural network is obtained by training using a back propagation algorithm according to the training data.

5. The mining image sensor system based on fusion RNN according to claim 1 is characterized in that: The mining image sensor system also includes a distributed analyzer module; The distributed analyzer module is used for supporting dynamic loading and unloading of the mining image sensor system based on a distributed analyzer in a factory mode.

6. The mining image sensor system based on fusion RNN according to claim 1 is characterized in that: The mining image sensor system also includes a multi-model fusion module; The multi-model fusion module is used to load and fuse the deep learning networks in sequence by using decision layer fusion to obtain a fused RNN model.

7. The mining image sensor system based on fusion RNN according to claim 6 is characterized in that: The decision-making layer is integrated into: Among them, Final Decision is the final decision result, argmax(·) represents the maximum value operation function, n represents the total number of candidate items, C i is a candidate, C is a decision candidate, δ(C i ,C) represents the function δ used to measure candidate C i The similarity or consistency with the current candidate C.

8. The mining image sensor system based on fusion RNN according to claim 6 is characterized in that: The mining image sensor system also includes an initialization calculation management module, a scheduling task management module, a feature recognition optimization module and a model update module; The initialization calculation management module is used to scan and register the device and its hardware resources to obtain the hardware status and task requirements of the device; The scheduling task management module is used to obtain NPU calculation results using the neural network processor according to the hardware status of the device and the task requirements; The feature recognition optimization module is used to perform feature recognition, weight adjustment and update on the deep learning model in sequence according to the NPU calculation result to obtain a feature recognition optimization result; The model updating module is used to realize continuous optimization and updating of the deep learning model by using online learning and meta-learning technology according to the feature recognition optimization result.

9. The mining image sensor system based on fusion RNN according to claim 8, characterized in that: The initialization calculation management module includes a system initialization unit and an algorithm initialization unit; The system initialization unit is used to scan and register the device to obtain the hardware status of the device; The algorithm initialization unit is used to scan and register the hardware resources of the device to obtain the task requirements of the device.

10. The mining image sensor system based on fusion RNN according to claim 1, characterized in that: The mining image sensor system also includes a cross-device communication module and a safety fault-tolerant module; The cross-device communication module is used for data transmission and communication across devices; The safety fault-tolerant module is used to provide a fault detection and recovery mechanism for the data transmission and communication.

Images