Coal production operation scene video AI algorithm training reasoning platform and method

Abstract

The invention discloses a coal production operation scene video AI algorithm training reasoning platform and method, and relates to the technical field of industrial safety monitoring, and the method comprises the steps: deploying a plurality of types of sensors, collecting environment data and video streams in real time, carrying out the denoising and timestamp calibration processing of the data, transmitting the processed data to a data processing center, and carrying out the video AI algorithm training reasoning of the coal production operation scene. Outputting an environment parameter data set; the comprehensive model is deployed on an edge computing device, real-time monitoring is performed on a coal production operation site, a video stream is subjected to model reasoning and is combined with environmental data to identify potential safety hazards, and a response is made in time through a deep reinforcement learning strategy. According to the method, the monitoring accuracy is improved through multi-sensor deployment and data preprocessing, the robustness and precision of the AI model are improved by utilizing enhancement processing and high-dimensional mapping, and the system can make a preventive decision by combining the adaptive learning rate and deep reinforcement learning, so that the safety and efficiency of coal production are effectively improved, and intelligent safety production is realized.

Description

Technical Field

[0001] The present invention relates to the field of industrial safety monitoring technology, and in particular to a coal production operation scene video AI algorithm training and reasoning platform and method. Background Art

[0002] In recent years, with the rapid development of Internet of Things (IoT) technology, artificial intelligence (AI) and edge computing, the intelligent transformation of the coal industry has become an industry consensus. The advancement of sensor technology has made real-time monitoring of environmental parameters possible, and the widespread use of high-definition video cameras has provided intuitive video monitoring for on-site operations. However, relying solely on manual monitoring and traditional statistical methods can no longer meet the growing needs of safety supervision and production optimization. AI algorithms, especially deep learning technology, have shown great potential in the application of coal production scenarios due to their powerful pattern recognition and decision support capabilities.

[0003] Although the application of AI and edge computing technologies in coal production operation scenarios has made certain progress, there are still some key challenges that limit their comprehensive promotion and in-depth application. First, traditional AI model training often relies on data in a fixed environment and lacks adaptability to dynamic environmental changes, which leads to limited generalization capabilities of the model in practical applications. Especially in a complex and changeable environment such as coal production, the robustness of the model is particularly important. Secondly, when processing video stream data, existing technologies usually ignore the real-time impact of environmental parameters, which makes the model unable to fully utilize environmental information for more accurate safety hazard identification and production optimization. Summary of the invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a coal production operation scenario video AI algorithm training and inference platform and method to solve the problems of limited model generalization ability and insufficient fusion of video stream and environmental parameters in dynamic environments.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In the first aspect, an embodiment of the present invention provides a video AI algorithm training and reasoning platform and method for coal production operation scenarios, which includes deploying multiple types of sensors, collecting environmental data and video streams in real time, denoising and calibrating timestamps on the data, transmitting the processed data to a data processing center, outputting an environmental parameter data set, creating a virtual environment using the environmental parameter data set, enhancing the original data through enhancement processing, using the enhanced original data to train an AI model, using an adaptive learning rate to allow the model to learn feature representations in different environments, constructing a deep reinforcement learning decision model based on coal production operation processes and target optimization, integrating the environmental adaptive AI model and the deep reinforcement learning decision model into a comprehensive model, deploying the comprehensive model on an edge computing device, and monitoring the coal production operation site in real time. The video stream is subjected to model reasoning and combined with environmental data to identify safety hazards, and then responds in a timely manner through a deep reinforcement learning strategy.

[0008] As a preferred solution of the coal production operation scene video AI algorithm training and inference platform and method described in the present invention, wherein: the deployment of multiple types of sensors, real-time collection of environmental data and video streams, denoising and calibrating timestamp processing of the data, transmitting the processed data to the data processing center, and outputting the environmental parameter data set, the specific steps are as follows:

[0009] According to the characteristics of the coal production environment, multiple types of sensors are installed at key locations on the working surface. After the sensors are deployed, environmental parameters and video stream data are collected in real time.

[0010] For the collected raw video data, a combination of median filtering and bilateral filtering is used to remove random noise and retain edge information, and the Kalman filtering algorithm is used to eliminate random fluctuations and outliers in the data;

[0011] Each frame of video and each piece of environmental data collected is timestamped and kept consistent with the synchronized clock. If clock deviation is detected, the timestamp is corrected using linear interpolation. The expression is:

[0012] t′=t+Δt;

[0013] Where Δt is the deviation between the local clock and the standard clock, t' is the corrected timestamp, and t represents the original timestamp;

[0014] The denoised and timestamp-calibrated data is encrypted using AES and transmitted to the data center, and stored in time series to form environmental parameter datasets and video datasets. The expression is:

[0015] D={d1,d2,…,d n};

[0016] Among them, d nrepresents the nth data point, and D represents a set of data points organized in time series.

[0017] As a preferred solution of the coal production operation scene video AI algorithm training and reasoning platform and method described in the present invention, wherein: the virtual environment is created by using the environmental parameter data set, and the environmental data is enhanced by enhancement processing, and the specific steps are as follows:

[0018] Extract key features from the environmental parameter data set D, including temperature T, humidity H, light intensity L, and dust concentration U, and construct a feature vector F = (T, H, L, U). Use a nonlinear mapping function to map the feature vector F to a high-dimensional feature space and enhance the feature vector. The expression is:

[0019]

[0020] Among them, α is the coefficient that controls the attenuation of light intensity, and ε is a small constant to prevent abnormal logarithmic operations. (F) is a high-dimensional feature space;

[0021] Design a set of environmental factor simulation functions, adjust the original video frames, and enhance the robustness and generalization ability of the model by simulating complex working environment conditions;

[0022] By adjusting the brightness of the image and simulating images taken under different lighting conditions, the model can learn the characteristics of objects in different lighting environments. The expression is:

[0023] B(L′)=L′*(1+β*sin(γ*L′));

[0024] Among them, β is used to control the brightness change amplitude, γ is the periodic adjustment factor, L′ represents the brightness value of the video frame after preliminary processing, and B(L′) is the adjusted brightness value;

[0025] Adjust the contrast of the image to highlight the key features in the image, making it easier for the model to capture these important distinguishing features. The expression is:

[0026] C(H′)=H′ / (1+δ*H ′2 );

[0027] Among them, 6 contrast control parameters, H′ represents the contrast value before adjustment or the environmental parameters related to the contrast, and C(H′) is the contrast after adjustment;

[0028] By adding random noise to the image, the model's recognition performance for interference images is improved. The expression is:

[0029] N(D′)=d+η*sqrt(D′)*∈;

[0030] Where d is the original pixel value, η is the noise intensity, ∈ is a random variable that obeys the standard normal distribution, D′ represents the dust concentration in the environment, and N(D′) represents the new pixel value after adding noise to the original pixel value.

[0031] For each original video frame V i Find the environmental parameters based on the corresponding timestamps. Based on the data mapped in the feature space, use the Kriging interpolation method to generate a continuous environmental parameter field to simulate the dynamic changes of the working environment.

[0032] The environmental parameter values ​​in the environmental parameter dataset D are matched to the environmental parameter vector F through the video frame timestamp i , F i Mapping to high-dimensional space (F i ), the video frame processed by the brightness adjustment function B, the contrast adjustment function C and the noise addition function N is combined with the environmental parameter vector mapped to the high-dimensional space to synthesize the enhanced video frame V′ i ;

[0033] V′ i Combined with the original environmental parameter data, an enhanced data set E = {V′1, V′2..., V′ i}.

[0034] As a preferred solution of the coal production operation scene video AI algorithm training and reasoning platform and method described in the present invention, the specific steps of using the enhanced raw data to train the AI ​​model are as follows:

[0035] The enhanced video frame V i ′ is input into the model, and the model outputs the predicted probability distribution Y pr , the expression is:

[0036] Y pr =f θ (V′ i );

[0037] Among them, f θ represents the model, and θ represents the model parameters;

[0038] Define a composite loss function, combine it with the prediction error and the environmental adaptation term, use the enhanced data as the input of the model, and introduce a weight factor ω to balance the importance of different loss parts. The expression is:

[0039] L=ω1·L pr (Y tr , Y pr )+ω2·L en (φ(F));

[0040] Among them, L pr is a common prediction loss function, L en Environmental fitness loss, Y pr , is the probability distribution of the model prediction output, Y tr The true label, is the high level representation of environmental factors, and ω1 is used to adjust the predicted loss L pr The relative importance of ω2 is used to control the environmental adaptability loss L en proportion.

[0041] As a preferred solution of the coal production operation scene video AI algorithm training and reasoning platform and method described in the present invention, wherein: the adaptive learning rate is used to allow the model to learn feature representations in different environments, and the specific steps are as follows:

[0042] Adopt the adaptive learning rate method based on gradient accumulation to dynamically adjust the learning rate η of each iteration to meet the training requirements of different environmental conditions. Use EMA to track the historical information of the gradient. The expression is:

[0043]

[0044] Where γ is the attenuation factor, Represents the modulus of the gradient of the current loss function with respect to the model parameters, and EMAgr is the current value of the exponential moving average;

[0045] By monitoring the smoothing index of historical gradient changes, the basic learning rate is dynamically modified adaptively. The expression is:

[0046]

[0047] Among them, η b is the basic learning rate, td is a threshold, and ηt is the learning rate at the current moment;

[0048] The learning rate is dynamically adjusted according to the set parameters, the composite loss function is used to guide the optimization, and the gradient information is updated through EMA, and the model parameters are gradually iterated until the training goal is reached.

[0049] As a preferred solution of the coal production operation scene video AI algorithm training and reasoning platform and method described in the present invention, the following is based on The mapped high-dimensional features and video features are used to build a deep reinforcement learning decision model. The specific steps are as follows:

[0050] Clarify the core elements of the deep reinforcement learning decision model and define the state space S, which consists of (F) The high-dimensional features and video features after mapping are expressed as:

[0051] S={s i |s i =[φ(F i ), V i ], i=1,...,N};

[0052] Among them, S i represents the i-th state in the state space, φ(F i ) represents the high-dimensional representation of the environmental parameter vector after transformation by the nonlinear mapping function φ, i = 1, ..., N represents N different states in the state space, V i The feature vector representing the video frame i;

[0053] Based on the state space S, a deep neural network is constructed as the policy network π(s; θ) and the value network V(s; θ v ), the policy network π(s; θ) outputs the probability distribution of taking each action under a given state, and the value network V(s; θ v ) evaluates the value of a given state, the policy network π(s; θ) and the value network V(s; θ v ) together constitute a deep reinforcement learning decision model, which is expressed as:

[0054] π(a|s;θ)=P(a|s);

[0055] Among them, π(a|s; θ) represents the policy probability distribution of taking action a in a given state S, P(a|s) represents the probability of taking action a in state S, and θ is the parameter set of the policy network;

[0056] V(s;θ v )=E a～π [R t |s t =s];

[0057] Among them, V(s; θ v ) represents the expected value of the future cumulative reward obtained by following the current strategy π in state s, E a～π Indicates that when action a follows the strategy π distribution, [R t |s t =s], indicating that starting from state s, the total reward R t The conditional expectation of v are the parameters of the value network.

[0058] As a preferred solution of the coal production operation scene video AI algorithm training and reasoning platform and method described in the present invention, wherein: the environment adaptive AI model and the deep reinforcement learning decision model are integrated into a comprehensive model, and the specific steps are as follows:

[0059] The environmental parameters predicted by the environment adaptive AI model Directly integrated into the state space S of the deep reinforcement learning decision model, the decision model can make more forward-looking decisions based on the current and predicted environmental states, expressed as:

[0060] S′=[φ(F c ),φ(F p ), V n ];

[0061] Among them, φ(Fp) is the prediction of the environmental parameters of the future moment by the environment adaptive AI model, S′ represents an enhanced state, and φ(F c ) represents the high-dimensional feature representation of the current environment parameters, V n Refers to the feature vector of the current video frame.

[0062] As a preferred solution of the coal production operation scene video AI algorithm training and reasoning platform and method described in the present invention, the comprehensive model is deployed on the edge computing device to monitor the coal production operation site in real time. The video stream is inferred by the model, combined with environmental data to identify safety hazards and respond in time through deep reinforcement learning strategies. The specific steps are as follows:

[0063] Use quantization to reduce model size and computational difficulty, and use Docker containerization to deploy the optimized model to edge computing devices.

[0064] Establish a complete process from video stream and environmental data collection to model inference. After the video stream is cropped, it is sent to the model for safety hazard identification. At the same time, the environmental data is matched with the video frame timestamp as part of the model input;

[0065] Design response logic based on the model reasoning results. For example, if safety hazards are discovered, the DRL strategy can be used to automatically adjust production line parameters to prevent potential accidents.

[0066] In a second aspect, an embodiment of the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, any step of the video AI algorithm training and inference platform and method for coal production operation scenarios as described in the first aspect of the present invention is implemented.

[0067] In a third aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, any step of the video AI algorithm training and inference platform and method for coal production operation scenarios as described in the first aspect of the present invention is implemented.

[0068] The beneficial effects of the present invention are as follows: by deploying multiple types of sensors and implementing precise data preprocessing, the present invention significantly enhances the accuracy and security of the data, laying the foundation for efficient monitoring in complex environments. The innovative enhanced processing of environmental parameter data sets, including high-dimensional mapping and environmental simulation, greatly improves the robustness and recognition accuracy of the AI ​​model under variable operating conditions. The AI ​​model training strategy combined with the adaptive learning rate not only accelerates the model's learning process of environmental characteristics, but also ensures the accuracy of the model's predictions and environmental adaptability. By intelligently integrating the environmental adaptive model with the deep reinforcement learning strategy, the system can make forward-looking decisions and effectively prevent safety hazards. Finally, the edge computing deployment of the model realizes real-time monitoring and rapid response of on-site video streams and environmental data, greatly improving the safety and operating efficiency of coal production, marking an important step towards intelligent and safe production. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0070] Figure 1 This is a flow chart of the video AI algorithm training and inference platform and method for the coal production operation scenario in Example 1.

[0071] Figure 2 This is the flow chart of environmental data collection and preprocessing in Example 1. DETAILED DESCRIPTION

[0072] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0073] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0074] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0075] Example 1, reference Figure 1 and Figure 2 , which is the first embodiment of the present invention, and provides a coal production operation scene video AI algorithm training and reasoning platform and method, including the following steps:

[0076] S1. Deploy multiple types of sensors to collect environmental data and video streams in real time, denoise the data and calibrate the timestamp, transmit the processed data to the data processing center, and output the environmental parameter data set.

[0077] Furthermore, according to the characteristics of the coal production environment, multiple types of sensors are installed at key locations on the working surface, including high-definition video cameras, temperature and humidity sensors, dust concentration detectors, and light intensity sensors. After the sensors are deployed, environmental parameters (such as temperature, humidity, light intensity, dust concentration) and video stream data are collected in real time (the video stream uses high-definition resolution to ensure clear image details);

[0078] For the collected raw video data, a combination of median filtering and bilateral filtering is used to remove random noise and retain edge information, and the Kalman filtering algorithm is used to eliminate random fluctuations and outliers in the data;

[0079] Each frame of video and each piece of environmental data collected is timestamped and kept consistent with the synchronized clock. If clock deviation is detected, the timestamp is corrected using linear interpolation. The expression is:

[0080] t′=t+△t;

[0081] Among them, △t is the deviation between the local clock and the standard clock, t′ is the corrected timestamp, and t represents the original timestamp;

[0082] The denoised and timestamp-calibrated data is encrypted using AES and transmitted to the data center, and stored in time series to form environmental parameter datasets and video datasets. The expression is:

[0083] D = {d1, d2, ..., d n};

[0084] Among them, d n represents the nth data point, and D represents a set of data points organized in time series.

[0085] It should also be noted that median filtering: This is a nonlinear filtering technique that is mainly used to remove salt and pepper noise. Its working principle is to apply a window to each pixel of the image and then replace the central pixel value with the median of all pixel values ​​within the window;

[0086] Bilateral filtering: This is an edge-preserving smoothing filter that keeps the edges of the image intact while filtering. Bilateral filtering not only considers the spatial proximity of pixels, but also the similarity of pixel values, filtering pixels through a weighted function, where the weight depends on the pixel distance and the difference in pixel values. This method can better preserve the structure and texture information of the image while removing noise.

[0087] Kalman filter: This is a recursive filtering algorithm used to estimate the state of a dynamic system, especially in a noisy environment. Kalman filtering can estimate the best estimate of the state of a linear dynamic system based on a series of noisy observations.

[0088] S2. Create a virtual environment using the environmental parameter data set and enhance the environmental data through enhancement processing.

[0089] Furthermore, key features, temperature T, humidity H, light intensity L and dust concentration U, are extracted from the environmental parameter data set D to construct a feature vector F = (T, H, L, U). The feature vector F is mapped to a high-dimensional feature space using a nonlinear mapping function to enhance the feature vector. The expression is:

[0090]

[0091] Among them, α is the coefficient that controls the attenuation of light intensity, and ε is a small constant to prevent abnormal logarithmic operations. (F) is a high-dimensional feature space,

[0092] Design a set of environmental factor simulation functions, adjust the original video frames, and enhance the robustness and generalization ability of the model by simulating complex working environment conditions;

[0093] By adjusting the brightness of the image and simulating images taken under different lighting conditions, the model can learn the characteristics of objects in different lighting environments. The expression is:

[0094] B(L′)=L′*(1+β*sin(γ*L′));

[0095] Among them, β is used to control the brightness change amplitude, γ is the periodic adjustment factor, L′ represents the brightness value of the video frame after preliminary processing, and B(L′) is the adjusted brightness value;

[0096] Adjust the contrast of the image to highlight the key features in the image, making it easier for the model to capture these important distinguishing features. The expression is:

[0097] C(H′)=H′ / (1+δ*H ′2 );

[0098] Among them, 6 contrast control parameters, H′ represents the contrast value before adjustment or the environmental parameters related to the contrast, and C(H′) is the contrast after adjustment;

[0099] By adding random noise to the image, the model's recognition performance for interference images is improved. The expression is:

[0100] N(D′)=d+η*sqrt(D′)*∈;

[0101] Where d is the original pixel value, η is the noise intensity, ∈ is a random variable that obeys the standard normal distribution, D′ represents the dust concentration in the environment, and N(D′) represents the new pixel value after adding noise to the original pixel value.

[0102] For each original video frame V i Find the environmental parameters based on the corresponding timestamps. Based on the data mapped in the feature space, use the Kriging interpolation method to generate a continuous environmental parameter field to simulate the dynamic changes of the working environment.

[0103] Combine V′i with the original environmental parameter data to form an enhanced data set E={V′1, V′2, . . . , V′m}.

[0104] It should also be noted that after processing the original data through feature space mapping and extracting key environmental features, the Kriging interpolation method is used to construct a continuous environmental parameter field based on the spatial correlation of the data. This process involves data preprocessing, variogram modeling, Kriging interpolation calculation, and dynamic change simulation. The parameter field finally generated can intuitively reflect the environmental dynamics and provide strong support for environmental management and decision-making. The verified model can be widely used in environmental monitoring, resource management and other fields, effectively improving the understanding and control capabilities of complex working environments.

[0105] S3. Use the enhanced raw data to train the AI ​​model and use an adaptive learning rate to allow the model to learn feature representations in different environments.

[0106] Furthermore, the enhanced video frame V i ′ is input into the model, and the model outputs the predicted probability distribution Y pr , the expression is:

[0107] Y pr =f θ (V′ i );

[0108] Among them, f θ represents the model, and θ represents the model parameters;

[0109] Define a composite function, combine it with the prediction error and the environmental adaptation term, use the enhanced data as the input of the model, and introduce a weight factor ω to balance the importance of different loss parts. The expression is:

[0110] L=ω1·L pr (Y tr , Y pr )+ω2·L en (φ(F));

[0111] Among them, L pr is a common prediction loss function, L en Environmental fitness loss, Y pr , is the model prediction output, Y tr The true label, is the high level representation of environmental factors, and ω1 is used to adjust the predicted loss L pr The relative importance of ω2 is used to control the environmental adaptability loss L en proportion.

[0112] Adopt the adaptive learning rate method based on gradient accumulation to dynamically adjust the learning rate η of each iteration to meet the training requirements of different environmental conditions. Use EMA to track the historical information of the gradient. The expression is:

[0113]

[0114] Where γ is the attenuation factor, Represents the modulus of the gradient of the current loss function with respect to the model parameters, EMA gr is the current value of the exponential moving average;

[0115] By monitoring the smoothing index of historical gradient changes, the basic learning rate is dynamically modified adaptively. The expression is:

[0116]

[0117] Among them, η b is the basic learning rate, td is a threshold, and ηt is the learning rate at the current moment;

[0118] The learning rate is dynamically adjusted according to the set parameters, the composite loss function is used to guide the optimization, and the gradient information is updated through EMA, and the model parameters are gradually iterated until the training goal is reached.

[0119] It should also be noted that in the optimization strategy, the adaptive learning rate method based on gradient accumulation is adopted to achieve efficient learning of model parameters. The exponential moving average (EMA) technology can smoothly track the historical gradient changes, and the attenuation factor is used to adjust the cumulative effect of the gradient, thereby dynamically updating the learning rate η. Specifically, in the adaptive learning rate expression, η b Represents the basic learning rate, and td is set as the threshold to ensure that the learning rate is adaptively adjusted with the training process to adapt to the training needs under different environmental conditions. Under the guidance of the composite loss function, the EMA technology is used to continuously optimize the gradient information and gradually iterate and update the parameters until the preset training goal is reached or the specified training cycle is completed, thereby demonstrating excellent generalization ability and prediction accuracy in a complex and changing environment.

[0120] S4, based on The mapped high-dimensional features and video features are used to build a deep reinforcement learning decision model, and the environment adaptive AI model and the deep reinforcement learning decision model are integrated into a comprehensive model.

[0121] Furthermore, we clarify the core elements of the deep reinforcement learning decision model and define the state space S and action space A. The state space S consists of After mapping, the high-dimensional features and video features, the action space includes production scheduling decisions (such as equipment operation speed, worker allocation, material supply, etc.), expressed as:

[0122] S={s i |s i =[φ(F i ), V i ], i=1,...,N};

[0123] Among them, S i represents the i-th state in the state space, φ(Fi) represents the high-dimensional representation of the environmental parameter vector after transformation by the nonlinear mapping function φ, i = 1, ..., N represents that there are N different states in the state space, V i The feature vector representing the video frame i;

[0124] Based on the state space S, a deep neural network is constructed as the policy network π(s; θ) and the value network V(s; θ v ), the policy network π(s; θ) outputs the probability distribution of taking each action under a given state, and the value network V(s; θ v ) evaluates the value of a given state, the policy network π(s; θ) and the value network V(s; θ v ) together constitute a deep reinforcement learning decision model, which is expressed as:

[0125] π(a|s;θ)=P(a|s);

[0126] Among them, π(a | s; θ) represents the policy probability distribution of taking action a in a given state S, P(a | s) represents the probability of taking action a in state S, and θ is the parameter set of the policy network;

[0127] V(s;θ v )=E a～π [R t |s t =s];

[0128] Among them, V(s; θ v ) represents the expected value of the future cumulative reward obtained by following the current strategy π in state s, E a～π Indicates that when action a follows the strategy π distribution, [R t |s t =s], indicating that starting from state s, the total reward R t The conditional expectation of v are the parameters of the value network.

[0129] The environmental parameters predicted by the environment adaptive AI model (F) is directly integrated into the state space S of the deep reinforcement learning decision model, so that the decision model can make more forward-looking decisions based on the current and predicted environmental states. The expression is:

[0130] S′=[φ(F c ),φ(F p ), V n ];

[0131] Among them, φ(Fp) is the prediction of the environmental parameters of the future moment by the environment adaptive AI model, S′ represents an enhanced state, and φ(F c ) represents the high-dimensional feature representation of the current environment parameters, V n Refers to the feature vector of the current video frame.

[0132] It should also be noted that the environmental adaptive AI model can predict the future trends of a series of key environmental parameters, such as temperature, humidity, light intensity, dust concentration, etc., by processing real-time sensor data and other environmental information. These prediction results φ(F) are encoded as high-dimensional feature vectors, which not only reflect the current environmental state, but also contain insights into the trend of environmental changes. In the deep reinforcement learning decision model, the state space S is traditionally composed of the currently observed environmental state and other task-related information. By integrating φ(F) into S, the decision model is not only based on the current environmental conditions, but also takes into account possible changes in environmental states in the future, so as to make more forward-looking decisions. Among them, φ(F) represents the prediction of environmental parameters, and the state space of the model is expanded to cover the prediction of future environmental changes. This allows the decision model to make more forward-looking decisions based not only on the current environmental conditions, but also on possible changes in environmental states in the future.

[0133] S5. Deploy the comprehensive model to the edge computing device to monitor the coal production site in real time. The video stream is inferred by the model, combined with environmental data to identify safety hazards and respond in a timely manner through deep reinforcement learning strategies.

[0134] Furthermore, we use quantization to reduce the model size and computational difficulty, adapt to the limited hard disk resources at the edge, and use Docker containerization to deploy the optimized model to the edge computing device.

[0135] Establish a complete process from video stream and environmental data collection to model inference. After the video stream is cropped, it is sent to the model for safety hazard identification. At the same time, the environmental data is matched with the video frame timestamp as part of the model input;

[0136] Design response logic based on the model reasoning results. For example, if safety hazards are discovered, the DRL strategy can be used to automatically adjust production line parameters to prevent potential accidents.

[0137] It should also be noted that Docker containerized deployment provides a unified, lightweight environment encapsulation, which packages the model and all the dependencies required for its operation together, ensuring that the model can run in a consistent and isolated environment regardless of development, testing or production environment, eliminating the common problem of "it can run on my machine". The image building process is defined through Dockerfile, including the installation of necessary libraries, copying of model files and service startup scripts. The final constructed Docker image is portable and efficient, and can be directly pushed to any hardware platform that supports Docker, including edge computing devices, to achieve one-click deployment, greatly simplifying the deployment process and improving the flexibility and reliability of deployment.

[0138] Combining Docker containerized deployment with model quantization processing not only solves the complexity and resource limitation problems of model deployment on edge computing devices, but also paves the way for the real-time application of AI models in coal production operation scenarios, achieving seamless connection from model training to deployment, ensuring the stable operation and efficient performance of the model in complex and changeable industrial environments, and providing solid technical support for the intelligent and safe coal production.

[0139] This embodiment also provides a computer device, which is suitable for the video AI algorithm training and reasoning platform and method for coal production operation scenarios, including: a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute computer executable instructions to implement the video AI algorithm training and reasoning platform and method for coal production operation scenarios as proposed in the above embodiments.

[0140] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covering the display screen, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0141] The present embodiment also provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements a video AI algorithm training and reasoning platform and method for coal production operation scenarios as proposed in the above embodiment; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, a disk or an optical disk.

[0142] In summary, by deploying multiple types of sensors and implementing precise data preprocessing, the present invention significantly improves the accuracy and security of the data on this platform, providing a solid foundation for efficient monitoring in complex environments. The innovative enhanced processing of environmental parameter data sets, including high-dimensional mapping and environmental simulation, greatly improves the robustness and recognition accuracy of the AI ​​model under variable operating conditions. The AI ​​model training strategy with adaptive learning rate accelerates the model's learning process of environmental characteristics and ensures the accuracy of the model's predictions and its environmental adaptability. By intelligently integrating the environmental adaptive model with the deep reinforcement learning strategy, the system can make forward-looking decisions and effectively prevent safety hazards. Finally, the edge computing deployment of the model realizes real-time monitoring and rapid response of on-site video streams and environmental data, greatly improving the safety and operating efficiency of coal production, marking an important step towards intelligent and safe production.

[0143] Example 2, referring to Table 1, is the second example of the present invention. To further verify the advancement of the present invention, experimental simulation data of the video AI algorithm training and reasoning platform and method for coal production operation scenarios are provided.

[0144] Four different test points were selected, and different temperature, humidity, light, dust, noise intensity, and video data characteristics were set at each test point.

[0145] Firstly, environmental data and video streams are collected through sensors and high-definition cameras, and denoising and timestamp calibration are performed to form high-quality environmental parameter datasets D and video datasets.

[0146] Secondly, according to the environmental parameter data set D, through the nonlinear mapping function By transforming features, combining Kriging interpolation and environmental simulation functions, the enhanced dataset E is constructed;

[0147] Next, using the enhanced dataset E, a composite loss function was defined and an adaptive learning rate method was adopted to train an AI model that can understand the complex coal production environment. The dynamic adjustment of the model parameters ensured rapid adaptation to the new environment.

[0148] Finally, a deep reinforcement learning decision model including state space S and action space A is constructed, and the prediction results of the environment adaptive AI model are integrated to form a comprehensive model S'. The model is quantified, containerized and deployed to edge computing devices, realizing real-time monitoring and intelligent response of the work site.

[0149] The details are shown in Table 1:

[0150] Table 1 Experimental record table

[0151]

[0152] Through the analysis of the data in the above table, compared with traditional monitoring technology, the present invention has significantly improved the environmental adaptability and decision-making intelligence level in coal production operations. For example, from test point 1 to test point 4, as the light intensity increases from 500 lux to 1200 lux, the model recognition accuracy rate not only does not decrease, but increases by 15%, which directly reflects the enhanced environmental adaptability through brightness and contrast adjustment.

[0153] Through the coal production operation scene video AI algorithm training and reasoning platform and method of the present invention, not only multi-dimensional and high-precision real-time monitoring of the operating environment is achieved, covering comprehensive perception from physical environment parameters to video streams, but also through deep learning and environmental adaptability optimization, the model's recognition and response capabilities under variable working conditions are significantly enhanced. When dealing with challenging conditions such as high dust concentrations and extreme light changes, it shows excellent stability and accuracy, reduces the risks of false alarms and missed alarms, and ensures the safe progress of production operations, reflecting the significant beneficial effects of the present invention in the field of coal production operation scene video AI algorithm training and reasoning.

[0154] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A coal production operation scene video AI algorithm training and reasoning platform and method, Features: include, Deploy multiple types of sensors to collect environmental data and video streams in real time, perform denoising and calibrate timestamps on the data, transmit the processed data to the data processing center, and output the environmental parameter data set; Using environmental parameter datasets to create virtual environments, the original data is enhanced through augmentation processing; Use the enhanced raw data to train the AI ​​model, and use an adaptive learning rate to allow the model to learn feature representations in different environments; Based on the coal production operation process and target optimization, a deep reinforcement learning decision model is constructed, and the environment adaptive AI model and the deep reinforcement learning decision model are integrated into a comprehensive model; The comprehensive model is deployed on edge computing devices to monitor the coal production site in real time. The video stream is inferred by the model and combined with environmental data to identify safety hazards, and then a deep reinforcement learning strategy is used to respond in a timely manner.

2. The coal production operation scene video AI algorithm training and reasoning platform and method as claimed in claim 1, Features: The method deploys multiple types of sensors, collects environmental data and video streams in real time, performs denoising and calibrates timestamp processing on the data, transmits the processed data to the data processing center, and outputs the environmental parameter data set. The specific steps are as follows: According to the characteristics of the coal production environment, multiple types of sensors are installed at key locations on the working surface. After the sensors are deployed, environmental parameters and video stream data are collected in real time. For the collected raw video data, a combination of median filtering and bilateral filtering is used to remove random noise and retain edge information, and the Kalman filtering algorithm is used to eliminate random fluctuations and outliers in the data; Each frame of video and each piece of environmental data collected is timestamped and kept consistent with the synchronized clock. If clock deviation is detected, the timestamp is corrected using linear interpolation. The expression is: t'=t+Δt; Where Δt is the deviation between the local clock and the standard clock, t' is the corrected timestamp, and t represents the original timestamp; The denoised and timestamp-calibrated data is encrypted using AES and transmitted to the data center, and stored in time series to form environmental parameter datasets and video datasets. The expression is: D={d 1 ,d 2 ,…,d n }; Among them, d n represents the nth data point, and D represents a set of data points organized in time series.

3. The coal production operation scene video AI algorithm training and reasoning platform and method as claimed in claim 2, Features: The virtual environment is created by using the environment parameter data set, and the environment data is enhanced by enhancement processing. The specific steps are as follows: Extract key features from the environmental parameter data set D, including temperature T, humidity H, light intensity L, and dust concentration U, and construct a feature vector F = (T, H, L, U). Use a nonlinear mapping function to map the feature vector F to a high-dimensional feature space and enhance the feature vector. The expression is: Among them, α is the coefficient that controls the attenuation of light intensity, and ε is a small constant to prevent abnormal logarithmic operations. It is a high-dimensional feature space; Design a set of environmental factor simulation functions, adjust the original video frames, and enhance the robustness and generalization ability of the model by simulating complex working environment conditions; By adjusting the brightness of the image and simulating images taken under different lighting conditions, the model can learn the characteristics of objects in different lighting environments. The expression is: B(L')=L'*(1+β*sin(γ*L')); Among them, β is used to control the brightness change amplitude, γ is the periodic adjustment factor, L' represents the brightness value of the video frame after preliminary processing, and B(L') is the adjusted brightness value; Adjust the contrast of the image to highlight the key features in the image, making it easier for the model to capture these important distinguishing features. The expression is: C(H')=H' / (1+δ*H' 2 ); Among them, δ is the contrast control parameter, H' represents the contrast value before adjustment or the environmental parameter related to the contrast, and C(H') is the contrast after adjustment; By adding random noise to the image, the model's recognition performance for interference images is improved. The expression is: N(D′)=d+η*Sqrt(D′)*∈; Where d is the original pixel value, η is the noise intensity, ∈ is a random variable that obeys the standard normal distribution, D' represents the dust concentration in the environment, and N(D') represents the new pixel value after adding noise to the original pixel value. For each original video frame V i Find the environmental parameters based on the corresponding timestamps. Based on the data mapped in the feature space, use the Kriging interpolation method to generate a continuous environmental parameter field to simulate the dynamic changes of the working environment. The environmental parameter values ​​in the environmental parameter dataset D are matched to the environmental parameter vector F through the video frame timestamp i , F i Mapping to high-dimensional space The video frame processed by the brightness adjustment function B, the contrast adjustment function C and the noise addition function N is combined with the environmental parameter vector mapped to the high-dimensional space to synthesize the enhanced video frame V' i ; V' i Combined with the original environmental parameter data, an enhanced data set E = {V' 1 ,V' 2 ...,V' i }.

4. The coal production operation scene video AI algorithm training and reasoning platform and method as claimed in claim 3, Features: The specific steps of using the enhanced original data to train the AI ​​model are as follows: The enhanced video frame V i 'Input into the model, the model outputs the predicted probability distribution Y pr , the expression is: Y pr =f θ (V′ i ); Among them, f θ represents the model, and θ represents the model parameters; Define a composite loss function, combine it with the prediction error and the environmental adaptation term, use the enhanced data as the input of the model, and introduce a weight factor ω to balance the importance of different loss parts. The expression is: L=ω 1 ·L pr (Y tr ,Y pr )+ω 2 ·L en (φ(F)); Among them, L pr is a common prediction loss function, L en Environmental fitness loss, Y pr , is the probability distribution of the model prediction output, Y tr The true label, is the high level representation of environmental factors, ω 1 Used to adjust the prediction loss L pr The relative importance of 2 Used to control environmental adaptability loss L en proportion.

5. The coal production operation scene video AI algorithm training and reasoning platform and method as claimed in claim 4, Features: The adaptive learning rate is used to allow the model to learn feature representations in different environments. The specific steps are as follows: Adopt the adaptive learning rate method based on gradient accumulation to dynamically adjust the learning rate η of each iteration to meet the training requirements of different environmental conditions. Use EMA to track the historical information of the gradient. The expression is: Where γ is the attenuation factor, Represents the modulus of the gradient of the current loss function with respect to the model parameters, EMA gr is the current value of the exponential moving average; By monitoring the smoothing index of historical gradient changes, the basic learning rate is dynamically modified adaptively. The expression is: Among them, η b is the basic learning rate, td is a threshold, and ηt is the learning rate at the current moment; The learning rate is dynamically adjusted according to the set parameters, the composite loss function is used to guide the optimization, and the gradient information is updated through EMA, and the model parameters are gradually iterated until the training goal is reached.

6. The coal production operation scene video AI algorithm training and reasoning platform and method as claimed in claim 5, Features: The basis The mapped high-dimensional features and video features are used to build a deep reinforcement learning decision model. The specific steps are as follows: Clarify the core elements of the deep reinforcement learning decision model and define the state space S, which consists of The high-dimensional features and video features after mapping are expressed as: S={s i |s i =[φ(F i ),V i ],i=1,...,N}; Among them, S i represents the i-th state in the state space, φ(F i ) represents the high-dimensional representation of the environmental parameter vector after transformation by the nonlinear mapping function φ, i=1,...,N represents that there are N different states in the state space, V i The feature vector representing the video frame i; Based on the state space S, a deep neural network is constructed as the policy network π(s; θ) and the value network V(s; θ v ), the policy network π(s; θ) outputs the probability distribution of taking each action under a given state, and the value network V(s; θ v ) evaluates the value of a given state, the policy network π(s; θ) and the value network V(s; θ v ) together constitute a deep reinforcement learning decision model, which is expressed as: π(a|s;θ)=P(a|s); Among them, π(a|s; θ) represents the policy probability distribution of taking action a in a given state S, P(a|s) represents the probability of taking action a in state S, and θ is the parameter set of the policy network; V(s;θ v )=E a～π [R t |s t =s]; Among them, V(s; θ v ) represents the expected value of the future cumulative reward obtained by following the current strategy π in state s, E a～π Indicates that when action a follows the strategy π distribution, [R t |s t =s], indicating that starting from state s, the total reward R t The conditional expectation of v are the parameters of the value network.

7. The coal production operation scene video AI algorithm training and reasoning platform and method as claimed in claim 6, Features: The specific steps of integrating the environment adaptive AI model and the deep reinforcement learning decision model into a comprehensive model are as follows: The environmental parameters predicted by the environment adaptive AI model Directly integrated into the state space S of the deep reinforcement learning decision model, the decision model can make more forward-looking decisions based on the current and predicted environmental states, expressed as: S'=[φ(F c ),φ(F p ),V n ]; Among them, φ(F p ), is the prediction of the environmental parameters of the future moment by the environment adaptive AI model, S' represents an enhanced state, φ(F c ) represents the high-dimensional feature representation of the current environment parameters, V n Refers to the feature vector of the current video frame.

8. The coal production operation scene video AI algorithm training and reasoning platform and method as claimed in claim 7, Features: The comprehensive model is deployed on the edge computing device to monitor the coal production site in real time. The video stream is inferred by the model, combined with environmental data to identify safety hazards and respond in a timely manner through deep reinforcement learning strategies. The specific steps are as follows: Use quantization to reduce model size and computational difficulty, and use Docker containerization to deploy the optimized model to edge computing devices. Establish a complete process from video stream and environmental data collection to model inference. After the video stream is cropped, it is sent to the model for safety hazard identification. At the same time, the environmental data is matched with the video frame timestamp as part of the model input; Design response logic based on the model reasoning results. For example, if safety hazards are discovered, the DRL strategy can be used to automatically adjust production line parameters to prevent potential accidents.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program. Features: When the processor executes the computer program, the steps of the coal production operation scene video AI algorithm training and reasoning platform and method as described in any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, Features: When the computer program is executed by a processor, the steps of the coal production operation scene video AI algorithm training and reasoning platform and method described in any one of claims 1 to 8 are implemented.

Images