Method and system for accurately controlling four carts of coke oven based on image recognition

Abstract

The invention relates to the technical field of industrial automation control, and discloses a coke oven four-cart accurate control method based on image recognition, which comprises the following steps: constructing a three-dimensional digital twin model of a coke oven environment; during online operation, a predicted visual angle image rendered by the model is utilized to guide a pre-trained generative correction model, and a distorted image collected by a camera in a severe environment is corrected into a high-fidelity image in real time. And fusing the corrected image, the thermal infrared image and the inertial data, and determining the current pose of the vehicle through Bayesian filtering. And continuously calculating a residual error between real observation and twinborn model prediction, identifying and quantifying systematic deviation caused by factors such as furnace body thermal deformation and the like, and asynchronously updating the three-dimensional digital twinborn model according to the systematic deviation. And performing closed-loop control on a vehicle operation execution mechanism based on the obtained high-precision pose. The method solves the problems that in the prior art, robustness is poor in a severe environment, and long-term deformation of the oven body cannot be adapted, and the accuracy and reliability of automatic operation of the coke oven are remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of industrial automation control technology, specifically to a method and system for precise control of the four main vehicles of a coke oven based on image recognition. Background Technology

[0002] The coke oven is the core thermal equipment in the coking industry, and the coke pusher, coal charging car, coke quenching car, and coke quenching car (collectively known as the "four major coke oven cars") are key mobile devices that operate in a cyclical manner around the coke oven. The reliability and accuracy of the automated operation of these four cars directly determine the efficiency, safety, and coke quality of the entire coking production process. Achieving precise and reliable automatic alignment between the four cars and their operational targets (such as the carbonization chamber opening) is the core and challenge of coke oven automation technology.

[0003] In existing technologies, methods such as laser ranging, encoder measurement, or visual recognition based on fixed markers are commonly used to achieve the positioning of the four main vehicles (coke ovens, trucks, and cranes). While these methods have promoted the automation of coke ovens to some extent, their performance and reliability are severely limited by the extremely harsh production environment of coke ovens in practical applications. High temperatures, large amounts of water vapor, and pervasive coal dust and coke powder are common in coke oven operating areas, all of which cause significant interference to positioning sensors. For example, high temperatures can cause uneven air density, leading to laser beam refraction, visual image distortion and blurring, and in severe cases, completely distorting the information output by the sensor. Simultaneously, water vapor and dust can directly obstruct the sensor's line of sight or contaminate the optical lens and pre-set physical markers, resulting in the loss of positioning information or significant errors.

[0004] Furthermore, existing positioning systems suffer from inherent limitations in environmental adaptability. Under prolonged high temperatures, coke oven bodies undergo irreversible thermal expansion and physical deformation, leading to slow but continuous changes in their geometry. Most existing positioning systems rely on one-time offline calibration, resulting in static environmental models. When the oven body deforms in the physical world, the static model no longer matches the real environment. This mismatch accumulates gradually, eventually causing a continuous decline in positioning accuracy. Solving this problem requires frequent manual shutdowns for inspection and recalibration, which not only increases maintenance workload and costs but also contradicts the goal of fully automated production. Therefore, existing technologies lack a self-learning, self-correcting mechanism capable of proactively sensing and adapting to changes in environmental physics, making it difficult to guarantee positioning accuracy and robustness over long periods of operation. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a precise control method and system for the four main cars of a coke oven based on image recognition. This solves the problems of poor robustness of existing positioning technologies for the four main cars of a coke oven under harsh environments such as high temperature and dust, and the inability to adapt to long-term physical deformation of the oven body, which leads to a decrease in accuracy.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a precise control method for the four main vehicles of a coke oven based on image recognition, comprising the following steps:

[0007] Obtain distorted images of the coke oven environment captured by cameras installed on the four main cars of the coke oven; input the distorted images into a pre-trained generative correction model to generate a corrected image;

[0008] Based on the corrected image and a pre-constructed three-dimensional digital twin model of the coke oven environment, the current pose of the four main coke oven cars is determined.

[0009] Based on the current pose, calculate the residual between the sensor observation data and the three-dimensional digital twin model; update the three-dimensional digital twin model according to the residual.

[0010] Preferably, the step of generating the corrected image further includes:

[0011] Based on the historical pose or motion information of the four coke oven cars, predict their prior pose.

[0012] Based on the prior pose, a predicted viewpoint image is rendered from the three-dimensional digital twin model;

[0013] The distorted image and the predicted viewpoint image are used as inputs and fed into the generative correction model to generate the corrected image.

[0014] Preferably, the generative correction model is a generative adversarial network; the generative adversarial network includes a generator and a discriminator, and is trained offline so that the generator can restore the distorted image to a high-fidelity image under the guidance of the predicted viewpoint image.

[0015] Preferably, the step of determining the current position of the four main coke oven cars further includes:

[0016] Acquire thermal infrared image data from a thermal infrared camera and inertial data from an inertial measurement unit;

[0017] Within the filtering framework, the corrected image, the thermal infrared image data, and the inertial data are fused to determine the current pose.

[0018] Preferably, the three-dimensional digital twin model includes geometric and thermodynamic information of the coke oven environment.

[0019] Preferably, the residuals include geometric residuals and thermodynamic residuals; the geometric residuals are used to characterize the structural differences between the corrected image and the geometric view rendered from the 3D digital twin model based on the current pose; the thermodynamic residuals are used to characterize the feature differences between thermal infrared image data and the thermodynamic view rendered from the 3D digital twin model based on the current pose.

[0020] Preferably, the step of updating the three-dimensional digital twin model is triggered only when the time series of the geometric residuals or the thermodynamic residuals are statistically analyzed and it is determined that there is a systematic bias.

[0021] Preferably, the condition for determining the systematic deviation is that the moving average of the time series of the residual is consistently higher than a preset mean threshold, and its variance is lower than a preset variance threshold.

[0022] Preferably, the method further includes: controlling the operating actuators on the four cars of the coke oven to perform precise operations based on the determined current pose.

[0023] The image recognition-based precision control system for the four main cars of a coke oven includes:

[0024] The image acquisition module is used to acquire distorted images of the coke oven environment captured by cameras installed on the four main cars of the coke oven;

[0025] An image correction module is used to input the distorted image into a pre-trained generative correction model to generate a corrected image;

[0026] The pose determination module is used to determine the current pose of the four main cars of the coke oven based on the corrected image and a pre-built three-dimensional digital twin model of the coke oven environment.

[0027] The residual calculation module is used to calculate the residual between the sensor observation data and the three-dimensional digital twin model based on the current pose.

[0028] The model update module is used to update the three-dimensional digital twin model based on the residual.

[0029] This invention provides a method and system for precise control of the four main cars of a coke oven based on image recognition. It has the following beneficial effects:

[0030] 1. This invention employs a pre-trained generative correction model to actively repair and reconstruct severely distorted images caused by high temperatures, heat waves, water vapor, and dust, fundamentally improving the information quality of visual sensors. Simultaneously, by combining thermal infrared image data, which is insensitive to smoke and lighting changes, as a supplementary positioning information source, redundancy and complementarity of multimodal information are achieved. This ensures that even under extreme conditions where information from a single sensor is severely damaged, the system can still stably output high-precision positioning results.

[0031] 2. This invention calculates the geometric and thermodynamic residuals between sensor observation data and a 3D digital twin model, and performs statistical analysis on the time series of these residuals. This enables the intelligent identification and learning of physical deformations or changes in thermal properties of the coke oven body caused by long-term high-temperature operation. Based on this, the system can asynchronously and precisely update the 3D digital twin model, ensuring it remains consistent with the real state of the physical world. This avoids the accuracy degradation and frequent manual calibration and maintenance problems caused by outdated models or damaged physical markers, common in traditional methods.

[0032] 3. This invention achieves deep collaboration through a virtuous cycle of bidirectional reinforcement. Specifically, the system uses the predicted viewpoint image rendered by the 3D digital twin model to guide the generative correction model, making image correction no longer a blind pixel-level repair, but a data-driven reconstruction that conforms to the geometric structure of the physical environment. The high-quality image after guided correction, in turn, provides a more reliable basis for pose determination and model residual calculation, further improving positioning accuracy. This deeply coupled design enables the overall system performance to surpass the simple combination of its parts, achieving higher-dimensional precise control. Attached Figure Description

[0033] Figure 1 This is a flowchart of the system method of the present invention. Detailed Implementation

[0034] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0035] Please see the appendix Figure 1 This invention provides a method and system for precise control of the four main cars of a coke oven based on image recognition. Before its online operation phase, the following offline preparation steps may be included:

[0036] S100: Construct the initial three-dimensional digital twin model.

[0037] S100: Training the generative correction model.

[0038] Step S100 is described in detail below. The goal of this step is to construct a high-fidelity initial three-dimensional digital twin model M0 containing both geometric and thermodynamic information. First, the geometric model is constructed. This process can be achieved by using a three-dimensional laser scanning device to comprehensively acquire three-dimensional point cloud data of the physical entity of the coke oven, or by directly utilizing an existing computer-aided design (CAD) model of the coke oven. Based on the acquired point cloud data or CAD model, a three-dimensional mesh model consisting of vertices, edges, and faces is generated through data processing. This model accurately reproduces the geometric shape and spatial topology of the coke oven body, columns, tracks, doors, and other key fixed facilities.

[0039] Based on the completed geometric model, thermodynamic information is assigned to it. This process is based on the established thermal process parameters of the coke oven, such as the coking cycle and the heating temperature curves at each stage, and combined with the known thermodynamic properties of the materials constituting different parts of the coke oven (such as the metal materials of the oven door and the refractory brick materials of the oven body), such as thermal conductivity, specific heat capacity, and thermal radiation coefficient. Numerical simulation methods such as Finite Element Analysis (FEA) are used to calculate the theoretical temperature values ​​of each node or region on the three-dimensional mesh model under steady-state or typical quasi-steady-state conditions. The calculated temperature field data is bound to the corresponding nodes of the three-dimensional mesh model to form the initial three-dimensional digital twin model M0. This model M0, as a queryable data structure, can return the geometric information and theoretical temperature information of a point based on the input three-dimensional coordinates.

[0040] Step S200 is described in detail below. The goal of this step is to train a generative correction model for insulating images affected by environmental disturbances in subsequent online stages. In one specific embodiment, the generative correction model is a generative adversarial network.

[0041] First, the training dataset is prepared. This dataset consists of a series of image pairs (I... distorted ,I clean It consists of ) . Among them, I distorted These are distorted images acquired from different positions and angles under normal coke oven production conditions, and affected by high-temperature heat waves, water vapor, or dust. clean Images are clear and free of geometric distortion, acquired under favorable observation conditions (e.g., during boiler shutdown for maintenance or when there is no heat wave interference) with the same or as similar camera poses. These images provide input and expected output for supervised training of the model.

[0042] Generative Adversarial Networks (GANs) structurally consist of a generator module G and a discriminator module D. In one embodiment, the generator module G adopts a U-Net architecture, which includes an encoder path and a decoder path. Feature maps from different levels in the encoder path are directly passed to the corresponding levels in the decoder path via skip connections. This structure helps preserve multi-scale details when generating images. The discriminator module D can adopt a PatchGAN architecture, where the output is not a single true / false probability value, but an NxN feature map. Each element in this feature map corresponds to the true probability of a local image patch in the input image. This structure helps the model focus on local texture and high-frequency details of the image.

[0043] The training process of the generator module G is a conditional generation process guided by a view rendered from a 3D digital twin model. Its input consists of two parts: the distorted image I to be corrected. distorted and a predicted viewpoint image I twinpred Predicted viewpoint image I twin_pred It is based on I clean The corresponding camera pose is an ideal geometric view rendered from the initial 3D digital twin model M0, without any distortion. During training, I... twinpred As a conditional input, with I distorted The feature maps are spliced ​​in the intermediate layers of the generator network or combined through other conditional fusion mechanisms to provide a definite geometric prior for the image correction process.

[0044] The training process of a generative adversarial network aims to solve for the combined loss function defined below:

[0045] G * =argmin G max D L cGAN (G,D)+λ L1 L L1 (G) where L cGAN (G,D) represents the conditional adversarial loss, and its mathematical expression is:

[0046]

[0047] This loss term, through an adversarial game between the generator and the discriminator, makes the distribution of images generated by the generator indistinguishable from that of real, clear images under given conditions.

[0048] L L1 (G) represents the L1 reconstruction loss, and its mathematical expression is:

[0049]

[0050] The algorithm calculates the image-by-image distance of the generated circular image to the target image. Constraining the generated image to maintain consistency with the target image helps generate sharper edges and structures.

[0051] This loss term directly calculates the pixel-wise L1 distance between the generated image and the real, clear image, constraining the generated image to maintain consistency with the target image in terms of content, which helps to generate clearer edges and structures.

[0052] In the above formula, the symbols are defined as follows:

[0053] G * : Indicates the generator module whose parameters have reached convergence after training.

[0054] G: indicates the generator module.

[0055] D: indicates the discriminator module.

[0056] λ L1 : Represents the weighting coefficient used to balance the conditional adversarial loss and the L1 reconstruction loss, which is a preset scalar value.

[0057] E: represents the mathematical expectation on the corresponding data distribution.

[0058] I distorted : Represents the distorted image input into the training dataset.

[0059] I clean : Represents the corresponding real, clear image in the training dataset.

[0060] I twinpred : Represents a conditioned predicted viewpoint image rendered by a 3D digital twin model.

[0061] ||.||: denotes the L1 norm.

[0062] The combined loss function is iteratively optimized using optimization algorithms such as Adam until the model converges. After training, the generator module G, whose parameters are already fixed, is then... * Save the data for later use in the online positioning phase.

[0063] After completing the offline preparation phase, the precise control method provided by this invention enters the online cyclic positioning and adaptive update phase. This phase is continuously executed during the operation of the four main coke oven cars. In one specific embodiment, the method may include the following steps:

[0064] S300: Acquires multimodal sensor data and performs motion state prediction.

[0065] S400: Generative correction of distorted images based on predicted viewpoint images.

[0066] S500: Determines the current pose by fusing multimodal observation data.

[0067] S600: Calculate geometric residuals and thermodynamic residuals.

[0068] S700: Asynchronous updates of 3D digital twin models based on residual time series analysis.

[0069] The following is a detailed explanation of step S300. At each time step t, the system first synchronously acquires multimodal data through sensors, specifically including: the distorted image of the current moment captured by the visible light camera. Thermal infrared image I acquired by thermal infrared camera at the current moment t hermal (t) ; and angular velocity and acceleration data U collected by the inertial measurement unit (IMU) within the time interval [t-1,t]. t°

[0070] Meanwhile, based on the optimal posterior pose X determined in the previous time step t-1 t-1|t-1 And the currently acquired inertial measurement unit data U t Using a pre-defined kinematic model f, the prior pose X at the current time step t is predicted. t |t-1. Its calculation process can be expressed as: X t|t-1 =f(X) t-1|t-1 U t The pose state vector X is a six-degree-of-freedom vector containing three-dimensional coordinates and three-dimensional pose information. This prior pose X... t|t-1 This provides an initial estimate for subsequent steps.

[0071] Step S400 will be described in detail below. This step utilizes the prior pose X obtained in step S300. t|t-1 And the current version of the 3D digital twin model M t-1 Through a graphics rendering function Π geo Render and generate a predicted viewpoint image without any distortion. Then, the predicted viewpoint image The distorted image acquired in real time in step S300 Both are used as input and fed into the generator module G, which has already been trained in the offline phase. * Generator module G * Output a corrected high-fidelity image I c orrected (t) The process can be represented as follows:

[0072]

[0073] The output of this step is I. c orrected (t) This provides high-quality visual information for subsequent pose determination.

[0074] Step S500 is described in detail below. This step is performed within a Bayesian filtering framework (e.g., Extended Kalman Filter, EKF) to fuse multimodal observation information. First, the corrected image output from step S400... Extract geometric features (such as corner points and straight lines of the furnace door frame) and extract them from the thermal infrared image acquired in step S300. Extract thermodynamic features (such as the contours or textures of specific high-temperature regions).

[0075] The extracted geometric and thermodynamic features are combined into an observation vector. Then, the predicted prior pose X from step S300 is used through an observation model. t|t-1 This is mapped to the observation space. Finally, within the filtering framework, by combining the actual observation vector and the predicted observation vector, the optimal posterior pose X at the current time step t is calculated. t|to .

[0076] The following is a detailed explanation of step S600. After determining the optimal posterior pose X at the current moment... t|t Then, the system calculates the actual sensor observation data and compares it with the current three-dimensional digital twin model M. t-1 The consistency difference between them is called the residual. In one specific embodiment, the residual includes geometric residuals and thermodynamic residuals.

[0077] Geometric residuals The calculation method is as follows:

[0078] In the context of SCTM (G), SSIM stands for Structural Similarity Index, used to measure the similarity between two images. geo (M t-1 ,X t|t ) represents the current optimal posterior pose X t |t from model M t-1 The rendered geometric view. The geometric residual R g eo quantifies the structural differences between the corrected real image and the v|t model view.

[0079] Thermodynamic residual The calculation method is as follows:

[0080]

[0081] Among them, Π therm From model M t-1 The function that renders the theoretical thermodynamic view in the middle; F thermo It is a feature extraction operator, such as Local Binary Pattern (LBP) or Histogram of Gradients (HOG), used to extract feature descriptors from thermal infrared images; ||.||2 denotes the L2 norm. The thermodynamic residual... The feature space distance between the actual heat distribution and the theoretical heat distribution of the model was quantified.

[0082] Step S700 is described in detail below. The goal of this step is to determine whether the 3D digital twin model needs to be updated based on the residual information. The system maintains a sliding time window of length N, storing the geometric residual sequence and thermodynamic residual sequence of the past N time steps.

[0083] At each time step t, the system performs statistical analysis on the residual sequence within the window. Model updates are triggered only when a systematic bias is identified. In one embodiment, the criteria for determining systematic bias are: the moving average of a residual sequence is consistently higher than a preset mean threshold, and its variance is lower than a preset variance threshold. Its logical expression can be represented as:

[0084]

[0085] in This represents the mean of the residual sequence. Let τ represent the variance of the residual sequence, and ∈ be the preset mean threshold and standard deviation threshold, respectively.

[0086] When the above conditions are met, the system performs a model update operation, updating the 3D digital twin model M based on the identified residual type and spatial distribution. t-1 By performing localized parameter adjustments (e.g., fine-tuning the coordinates of some mesh vertices, or correcting the thermal properties of the material in specific regions), a new version of model M is generated. t If the above conditions are not met, then the update will not be performed, and M will be set to M. t =M t-1 This asynchronous update mechanism ensures the stability of the model and enables it to adapt to real changes in the physical world.

[0087] In one specific embodiment, this step can be further broken down into the following sub-steps:

[0088] S810: Obtain the target pose corresponding to the task and perform coordinate system registration.

[0089] S820: Real-time calculation of the six-degree-of-freedom deviation between the current pose and the target pose.

[0090] S830: Based on the deviation, control commands are generated through a control algorithm.

[0091] S840: Drives the actuator to respond to control commands to achieve precise alignment.

[0092] In sub-step S810, the onboard control system of the four main coke oven cars first queries and obtains the target pose X that strictly corresponds to the current operation instruction (e.g., pushing coke into the Nth carbonization chamber) from a preset task database. target Target pose X target It is a pre-calibrated and stored six-degree-of-freedom vector that defines the ideal spatial position and attitude that the vehicle must achieve when the actuators perform critical tasks. Simultaneously, the system performs coordinate system registration to ensure that the current pose X, output by the positioning method, is in the global reference coordinate system. t|t , and the target pose X target The local coordinate systems of each operating actuator on the vehicle are under a unified coordinate reference.

[0093] In sub-step S820, the vehicle control system continuously updates the current optimal posterior pose X determined in step S500. t|t The target pose X obtained in sub-step S810 target Compare the two and calculate the six-degree-of-freedom deviation vector ΔX between them.

[0094] ΔX=X target -X t|t =[Δp x ,Δp y ,Δp z ,Δφ,Δθ,Δψ] T ;

[0095] Where, Δp x ,Δp y ,Δp z Δφ, Δθ, and Δψ represent the vehicle's positional deviations in the track's forward, lateral, and vertical directions, respectively; Δφ, Δθ, and Δψ represent the vehicle's attitude deviations in roll, pitch, and yaw angles, respectively. This deviation vector ΔX precisely quantifies the difference between the vehicle's current state and its ideal operating state. In sub-step S830, the deviation vector ΔX is fed as input into one or more preset controllers. In one embodiment, this controller is a proportional-integral-derivative (PID) controller. The PID controller adjusts the input based on the deviations of each component (e.g., Δp). y Δp z (etc.), and its

[0096] The rate of change and cumulative amount over time are used to calculate one or more sets of control commands through a preset control law. These control commands are specific physical execution signals, such as the target speed or torque of the motor driving the walking mechanism, the opening degree or pressure of the hydraulic valve, or the target position of the servo motor.

[0097] In sub-step S810, the onboard control system of the four main coke oven cars first queries and obtains the target pose X that strictly corresponds to the current operation instruction (e.g., pushing coke into the Nth carbonization chamber) from a preset task database. target Target pose X target It is a pre-calibrated and stored six-degree-of-freedom vector that defines the ideal spatial position and attitude that the vehicle must achieve when the actuators perform critical tasks. Simultaneously, the system performs coordinate system registration to ensure that the current pose X, output by the positioning method, is in the global reference coordinate system. t|t , and the target pose X target The local coordinate systems of each operating actuator on the vehicle are under a unified coordinate reference.

[0098] In sub-step S820, the vehicle control system continuously updates the current optimal posterior pose X determined in step S500. t|t The target pose X obtained in sub-step S810 target Compare the two and calculate the six-degree-of-freedom deviation vector ΔX between them.

[0099] ΔX=X target -X t|t =[Δp x ,Δp y ,Δp z ,Δφ,Δθ,Δψ] T ;

[0100] Where, Δp x ,Δp y ,Δp z Δφ, Δθ, and Δψ represent the vehicle's positional deviations in the forward, lateral, and vertical directions along the track, respectively; Δφ, Δθ, and Δψ represent the vehicle's attitude deviations in roll, pitch, and yaw angles, respectively. This deviation vector ΔX precisely quantifies the difference between the vehicle's current state and its ideal operating state. In sub-step S830, the deviation vector ΔX is fed as input into one or more preset controllers. This controller is a proportional-integral-derivative (PID) controller. The PID controller adjusts the input based on the deviations of each component (e.g., Δp). y Δp z (etc.), and its

[0101] The rate of change and cumulative amount over time are used to calculate one or more sets of control commands through a preset control law. These control commands are specific physical execution signals, such as the target speed or torque of the motor driving the walking mechanism, the opening degree or pressure of the hydraulic valve, or the target position of the servo motor.

[0102] In sub-step S840, the control commands generated in sub-step S830 are sent to the driver of the corresponding operating mechanism. The driver responds to these commands, driving the mechanism to perform physical movement to eliminate the pose deviation calculated in sub-step S820.

[0103] Taking the coke pusher as an example: If the calculated lateral position deviation Δp y If the yaw angle deviation Δψ is not zero, the control command will drive the coke pusher's traveling mechanism to make fine adjustments, and may also drive the coke pusher's centering mechanism to make lateral micro-movements, until Δp y The alignment and Δψ converge to a preset allowable error range (e.g., less than 5 mm and 0.1 degrees). Only after alignment is completed does the onboard control system issue the next command, allowing the focus pusher to extend.

[0104] Taking the coke chute as an example: If the calculated lateral position deviation Δp y and vertical position deviation Δp z If the value is not zero, the control command will drive the coke quenching car's traveling mechanism and the coke quenching knife lifting mechanism to operate simultaneously or separately, so that the centerline of the coke quenching knife is precisely aligned with the centerline of the furnace door on the coke side of the target carbonization chamber. This process continues until all components of the deviation vector ΔX meet the operational accuracy requirements.

[0105] Through the closed-loop control sub-steps S810 to S840 described above, the present invention transforms high-precision positioning information into actual and precise physical control actions, thereby ensuring the reliability and accuracy of the automated operation of the four main coke oven cars.

[0106] This invention also provides a precision control system for the four main cars of a coke oven based on image recognition. In one embodiment, the system can be deployed on the four main cars of the coke oven, or some modules can be deployed on the cars and some modules can be deployed on a remote server. The system may include: an image acquisition module, an image correction module, a pose determination module, a residual calculation module, and a model update module.

[0107] In one embodiment, the system may further include a data storage module and a control execution module.

[0108] The image acquisition module is used to acquire distorted images of the coke oven environment captured by cameras installed on the four main coke oven vehicles. Specifically, this module may include one or more hardware devices, such as a visible light industrial camera and a thermal infrared camera, as well as an inertial measurement unit. The module is configured to synchronously acquire distorted visible light images, thermal infrared images, and vehicle angular velocity and acceleration data at a preset frequency, and provide this multimodal sensor data to other modules in the system.

[0109] The image correction module takes a distorted image as input and feeds it into a pre-trained generative correction model to generate a corrected image. This module connects to the pose determination module, first obtaining the predicted prior pose from the pose determination module. Then, based on this prior pose, the module renders a predicted viewpoint image from a 3D digital twin model stored in the data storage module. Finally, the module calls a pre-trained generator model stored in the data storage module, taking the distorted image acquired by the image acquisition module and the rendered predicted viewpoint image as input, and outputs a high-fidelity corrected image.

[0110] The pose determination module determines the current pose of the four main vehicles of the coke oven based on a corrected image and a pre-built 3D digital twin model of the coke oven environment. This module receives inertial data and thermal infrared images from the image acquisition module, as well as a corrected image from the image correction module. Internally, this module integrates a Bayesian filtering algorithm. It first predicts the motion state based on the inertial data to obtain the prior pose. Then, it fuses geometric features from the corrected image and thermodynamic features from the thermal infrared image as observation information to update the prior pose, and finally calculates and outputs the vehicle's current optimal posterior pose.

[0111] The residual calculation module calculates the residual between sensor observation data and the 3D digital twin model based on the current pose. This module obtains the current optimal posterior pose from the pose determination module and acquires corrected images and thermal infrared images from the image correction module and image acquisition module, respectively. Based on the optimal posterior pose, this module renders the corresponding geometric and thermodynamic views from the 3D digital twin model in the data storage module, then compares them with the corrected and thermal infrared images to calculate the geometric and thermodynamic residuals, and outputs the calculation results to the model update module.

[0112] The model update module updates the 3D digital twin model based on the residuals. This module receives time series data of geometric and thermodynamic residuals from the residual calculation module. Internally, it includes a logic unit for determining systematic biases, which performs statistical analysis on the residual series. When the determination criteria are met, the module is triggered and, based on the type and distribution of the residuals, locally adjusts the parameters of the 3D digital twin model stored in the data storage module to generate a new version of the model.

[0113] A data storage module is used to persistently store various types of data and models required for system operation. In one embodiment, this module can be a non-volatile memory. Its stored content includes at least: initial and all updated versions of the 3D digital twin model, parameters of the pre-trained generative correction model, pose data generated during system operation, residual data, and various preset threshold parameters.

[0114] The control execution module is used to control the operating actuators on the four main cars of the coke oven to perform precise operations based on the current pose output by the pose determination module. This module receives the optimal posterior pose output by the pose determination module and compares it with the target operating pose retrieved from the data storage module to calculate the pose deviation. Then, the module generates control commands through a control algorithm and sends these commands to the drivers of the car's traveling mechanism, centering mechanism, lifting mechanism, etc., via interface circuits, thus forming a closed-loop control.

[0115] In the actual hardware implementation of the system, the functions of the aforementioned image correction module, pose determination module, residual calculation module, model update module, and control execution module can be implemented by one or more processors executing computer program instructions stored in the data storage module. These modules interact and work together via an internal bus or network to complete the precise control method proposed in this invention.

[0116] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A precise control method for the four main cars of a coke oven based on image recognition, characterized in that, Includes the following steps: Obtain distorted images of the coke oven environment captured by cameras installed on the four main cars of the coke oven; The distorted image is input into a pre-trained generative correction model to generate a corrected image; Based on the corrected image and a pre-constructed three-dimensional digital twin model of the coke oven environment, the current pose of the four main coke oven cars is determined. Based on the current pose, calculate the residual between the sensor observation data and the three-dimensional digital twin model; The three-dimensional digital twin model is updated based on the residual.

2. The precise control method for the four main cars of a coke oven based on image recognition according to claim 1, characterized in that, The step of generating the corrected image further includes: Based on the historical pose or motion information of the four coke oven cars, predict their prior pose. Based on the prior pose, a predicted viewpoint image is rendered from the three-dimensional digital twin model; The distorted image and the predicted viewpoint image are used as inputs and fed into the generative correction model to generate the corrected image.

3. The precise control method for the four main cars of a coke oven based on image recognition according to claim 2, characterized in that, The generative correction model is a generative adversarial network; the generative adversarial network includes a generator and a discriminator, and through offline training, the generator is able to restore the distorted image to a high-fidelity image under the guidance of the predicted viewpoint image.

4. The precise control method for the four main cars of a coke oven based on image recognition according to claim 1, characterized in that, The step of determining the current position of the four main cars of the coke oven further includes: Acquire thermal infrared image data from a thermal infrared camera and inertial data from an inertial measurement unit; Within the filtering framework, the corrected image, the thermal infrared image data, and the inertial data are fused to determine the current pose.

5. The precise control method for the four main cars of a coke oven based on image recognition according to claim 1, characterized in that, The three-dimensional digital twin model includes the geometric and thermodynamic information of the coke oven environment.

6. The precise control method for the four main cars of a coke oven based on image recognition according to claim 1, characterized in that, The residuals include geometric residuals and thermodynamic residuals; the geometric residuals are used to characterize the structural differences between the corrected image and the geometric view rendered from the 3D digital twin model based on the current pose. The thermodynamic residual is used to characterize the feature differences between thermal infrared image data and the thermodynamic view rendered from the three-dimensional digital twin model based on the current pose.

7. The precise control method for the four main cars of a coke oven based on image recognition according to claim 6, characterized in that, The step of updating the three-dimensional digital twin model is triggered only when the time series of the geometric residuals or the thermodynamic residuals are statistically analyzed and it is determined that there is a systematic deviation.

8. The precise control method for the four main cars of a coke oven based on image recognition according to claim 7, characterized in that, The condition for determining the systematic bias is that the moving average of the time series of the residuals is consistently higher than a preset mean threshold, and its variance is lower than a preset variance threshold.

9. The precise control method for the four main cars of a coke oven based on image recognition according to claim 1, characterized in that, The method also includes: controlling the operating actuators on the four cars of the coke oven to perform precise operations based on the determined current pose.

10. A precision control system for the four main cars of a coke oven based on image recognition, comprising the precision control method for the four main cars of a coke oven based on image recognition according to any one of claims 1-9, characterized in that, include: The image acquisition module is used to acquire distorted images of the coke oven environment captured by cameras installed on the four main cars of the coke oven; An image correction module is used to input the distorted image into a pre-trained generative correction model to generate a corrected image; The pose determination module is used to determine the current pose of the four main cars of the coke oven based on the corrected image and a pre-built three-dimensional digital twin model of the coke oven environment. The residual calculation module is used to calculate the residual between the sensor observation data and the three-dimensional digital twin model based on the current pose. The model update module is used to update the three-dimensional digital twin model based on the residual.

Images