A device control method and system

Abstract

This application provides a device control method and system, relating to the field of dry quenching technology. The method includes: in response to device status detection conditions meeting a target stage of the dry quenching process, acquiring target images of each detection area under the target stage; wherein each detection area includes: an area containing a designated component in the target device to be monitored in the target stage; performing status detection on the designated component in the target image of each detection area to obtain a status detection result for the designated component; determining the status detection result of the target device in the target stage based on the obtained status detection result; and performing interlocking control on the target device based on the status detection result of the target device. This application enables more accurate interlocking control of the device.

Description

Technical Field

[0001] This application relates to the field of dry quenching technology, and in particular to a method and system for controlling equipment. Background Technology

[0002] Dry quenching is an energy-saving and environmentally friendly technology that uses circulating inert gas to cool red-hot coke in a closed system.

[0003] In the dry quenching process, the elevator is responsible for lifting the coke cans loaded with red coke to the top of the tower, moving them horizontally, lowering them to complete the coke loading, and then returning them to the empty cans. The operational safety of the equipment involved in the dry quenching process is directly related to the safety and stability of the dry quenching process.

[0004] In related technologies, the status of equipment in the dry quenching process is confirmed by a single switch signal, and then the equipment is controlled based on the confirmed status. For example, if the switch signal is a hook open signal, then detecting this signal indicates that the hook of the hoist and the lifting lug of the coke pot are in a closed state.

[0005] However, the amount of information contained in a single switch signal is small, making it difficult to accurately represent the state of the equipment, and consequently, difficult to accurately control the equipment. Summary of the Invention

[0006] The purpose of this application is to provide a device control method and system for more accurate interlocking control of devices. The specific technical solution is as follows:

[0007] In a first aspect, embodiments of this application provide a device control method, the method comprising:

[0008] In response to the equipment status detection conditions for meeting the target stage of the dry quenching process, target images of each detection area under the target stage are acquired; wherein, each detection area includes: the area where a specified component is located in the target equipment that needs to be focused on in the target stage;

[0009] Perform state detection on specified components in the target image of each detection area to obtain the state detection results of the specified components;

[0010] Based on the obtained status detection results, determine the status detection results of the target device in the target stage;

[0011] Based on the status detection results of the target device, interlock control is performed on the target device.

[0012] Optionally, the step of performing state detection on specified components in the target image of each detection region to obtain the state detection result of the specified components includes:

[0013] For each detection area, the state description information of a specified component in the target image is obtained.

[0014] Based on the state description information, determine the state detection result of the specified component.

[0015] Optionally, for each detection region, obtaining state description information of a specified component in the target image includes:

[0016] For each detection area of ​​the target image, the positions of multiple specified components in the target image are determined;

[0017] Based on the determined locations, determine the target geometric relationships of multiple specified components in the target image;

[0018] Determining the state detection result of the specified component based on the state description information includes:

[0019] Based on the first correspondence between the geometric relationship and the state of multiple specified components in the detection area, the state of the target component corresponding to the target geometric relationship is determined, and the first state detection result of the specified component is obtained.

[0020] And / or, for each detection region, obtaining state description information of a specified component in the target image includes:

[0021] For each detection region's target image, determine the target pose information of a specified component in the target image;

[0022] Determining the state detection result of the specified component based on the state description information includes:

[0023] Based on the target pose information, determine the target contour of the specified component in the target image;

[0024] Determine whether the target contour is within the boundary range of a designated component in the pre-set detection area. If yes, determine that the designated component has not crossed the boundary; otherwise, determine that the designated component has crossed the boundary, and obtain the second state detection result of the designated component.

[0025] And / or, for each detection region, obtaining state description information of a specified component in the target image includes:

[0026] For each detection region of the target image, determine the component detection region of the specified component in the target image;

[0027] The state of the specified component is classified according to the component detection area to obtain the state classification result of the specified component.

[0028] Determining the state detection result of the specified component based on the state description information includes:

[0029] Based on the state classification result of the component, determine the third state detection result of the specified component.

[0030] Optionally, the target image of any detection area includes: multiple consecutive frames of images acquired for the detection area; the step of performing state detection on a specified component in the target image of each detection area to obtain the state detection result of the specified component includes:

[0031] Perform state detection on a specified component in multiple consecutive frames of images for each detection region to obtain the detection results for each of the multiple consecutive frames.

[0032] Based on the detection results of each of the consecutive frames of images, determine the state detection result of the specified component;

[0033] The methods for performing state detection on a specified component in any frame of an image include:

[0034] Determine the detection result of a specified state of a specified component in the frame image; wherein the specified state detection result includes at least one of the following: the first state detection result, the second state detection result, and the third state detection result;

[0035] Based on the specified state detection result, the state detection result of the specified component is determined.

[0036] Optionally, for each detection region, obtaining state description information of a specified component in the target image includes:

[0037] For each detection region of the target image, the state detection model corresponding to that detection region is called to perform state detection on the specified component in the target image and obtain the state description information of the specified component.

[0038] The training methods for the state detection model corresponding to any detection region include:

[0039] Acquire multiple initial sample images of the detection area; wherein each initial sample image contains a specified component in the detection area;

[0040] The state detection model to be trained in the detection region is invoked to perform state detection on multiple initial sample images, and the initial state detection results of each initial sample image are obtained.

[0041] Based on the initial state detection results of each initial sample image, determine the state calibration true value of each initial sample image;

[0042] Based on the state calibration ground values ​​of each initial sample image, the state detection model to be trained is trained to obtain the state detection model corresponding to the detection region after training.

[0043] Optionally, the method further includes:

[0044] For any detection area corresponding to the state detection model, the state detection true value of the specified component is determined based on the state description information of the specified component obtained from the state detection model.

[0045] The state detection model is updated and trained based on the state detection ground truth.

[0046] Optionally, the step of interlocking the target device based on the status detection result of the target device includes:

[0047] If the status detection result of the target device indicates that the target device is in a normal state, an interlock control signal is sent to the target device to control the target device to perform the action of the target stage; otherwise, a prohibition signal is sent to the target device to prohibit the action of the target device.

[0048] Secondly, embodiments of this application provide a device control system, the system comprising: an image acquisition unit, a recognition unit, and a control unit;

[0049] The image acquisition unit is configured to acquire target images of each detection area during the target stage; wherein, each detection area includes: the area where a specified component is located in the target device that needs to be focused on during the target stage;

[0050] The control unit is configured to, in response to the equipment status detection conditions of the target stage of the dry quenching process, acquire target images of each detection area under the target stage from the image acquisition unit; and send the target images of each detection area under the target stage to the recognition unit.

[0051] The identification unit is configured to receive target images of each detection area under the target stage, perform state detection on a specified component in the target image of each detection area, obtain the state detection result of the specified component, and feed back the state detection result of the specified component to the control unit.

[0052] The control unit is further configured to receive the status detection result of a designated component, determine the status detection result of the target device in the target stage based on the obtained status detection result, and perform interlocking control on the target device based on the status detection result of the target device.

[0053] Optionally, the system further includes: an interlocking control interface module; the interlocking control interface module is provided with interfaces that need to be called when performing interlocking control at each stage;

[0054] The control unit performs interlocking control on the target device based on the status detection result of the target device, including:

[0055] If the status detection result of the target device indicates that the target device is in a normal state, then the target interlocking control interface corresponding to the target stage in the interlocking control interface unit is invoked to send an interlocking control signal to the target device to control the target device to perform the action of the target stage; otherwise, the prohibition interface corresponding to the target stage in the interlocking control interface unit is invoked to send a prohibition signal to the target device to prohibit the action of the target device.

[0056] Optionally, the system further includes a lighting unit, an alarm unit, a display unit, and a data storage unit;

[0057] The illumination unit is configured to provide a light source when the image acquisition unit acquires target images of each detection area during the target stage.

[0058] The alarm unit is configured to issue an alarm signal when the status detection result of the target device indicates that the target device is not in a normal state;

[0059] The display unit is configured to acquire target images of each detection area under the target stage from the image acquisition unit, acquire the status detection results of a specified component from the recognition unit, acquire the alarm event represented by the alarm signal from the alarm unit, acquire the status detection results of the target device from the control unit, and display the acquired content.

[0060] The storage unit is configured to acquire target images of each detection area under the target stage from the image acquisition unit, acquire the status detection results of a specified component from the recognition unit, acquire the alarm event represented by the alarm signal from the alarm unit, acquire the status detection results of the target device from the control unit, and store the acquired content.

[0061] Beneficial effects of the embodiments in this application:

[0062] The equipment control method provided in this application, when meeting the equipment status detection conditions of the target stage of the dry quenching process, acquires target images of each detection area where a specified component of the target equipment of interest in the target stage is located; then, by performing status detection on the specified component in the target image of each detection area, the status detection result of the specified component is obtained, and the status detection result of the target equipment in the target stage is determined accordingly; thus, the status detection result of the target equipment includes the status detection result of the specified component in the target equipment, and based on the status detection result of the target equipment, more accurate interlocking control of the equipment can be performed.

[0063] Of course, implementing any product or method of this application does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0064] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0065] Figure 1 A flowchart illustrating a device control method provided in an embodiment of this application;

[0066] Figure 2 This is a schematic diagram illustrating a process for detecting the state of a specified component, as provided in an embodiment of this application.

[0067] Figure 3 A flowchart illustrating a training method for a state detection model provided in an embodiment of this application;

[0068] Figure 4 This is a schematic diagram of the structure of a device control system provided in an embodiment of this application;

[0069] Figure 5 A hierarchical schematic diagram of a device control system provided in an embodiment of this application;

[0070] Figure 6 This is a block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0071] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of this application.

[0072] In the dry quenching process, the elevator plays a crucial role in lifting, hoisting, and transferring coke cans. The operational safety of the equipment involved in the dry quenching process directly affects the safety and stability of the process. In related technologies, the status of equipment in the dry quenching process is typically determined by manual observation, ordinary video monitoring, or single-switch signals at the site. For example, this includes checking whether the elevator hook is closed or has exceeded its limits, and whether the bottom gate of the coke can is clamped with coke.

[0073] The above-mentioned traditional solutions have at least the following shortcomings:

[0074] First, manual observation relies heavily on human operational experience, and under complex lighting conditions such as night, dust, backlight, rain, and snow, it is easy to miss or misjudge the equipment status.

[0075] Secondly, the information content of a single switch signal is small, making it difficult to accurately reflect the status of the equipment (e.g., the hook open position signal can only indicate that the hook is open in place, but cannot indicate whether the hook is fully closed or whether the hook is actually in contact with the lifting lug). It has poor adaptability to scenarios with partial obstruction, changes in mechanical posture, and on-site pollution.

[0076] Third, existing ordinary video surveillance is mostly "visible but uncontrollable". Even if abnormal equipment status is detected, it is difficult to interlock with the hoist's PLC (Programmable Logic Controller) and locomotive in a timely manner, and cannot automatically stop the equipment from operating under dangerous conditions. In the current situation, it is usually necessary to manually issue subsequent interlock control commands based on the equipment status, such as manually clicking the interlock control button, etc.

[0077] Fourth, on-site alarms, recording, tracing, model training (such as training a model to analyze the state of a certain component) and online model optimization are usually scattered in different systems, resulting in insufficient data closure, which makes it difficult to maintain the model and continuously improve the model's accuracy through optimization training.

[0078] Therefore, there is an urgent need for an intelligent system and method that can stably identify the equipment status under complex operating conditions, output the identification results in real time and form a safety interlock, while also taking into account alarm, traceability, data annotation and model iteration capabilities.

[0079] Based on this, embodiments of this application provide a device control method and system for accurately controlling the device.

[0080] The following describes a device control method provided by an embodiment of this application.

[0081] One of the equipment control methods provided in this application can be applied to electronic devices, such as mobile phones and computers, which can perform status analysis on equipment in the dry quenching process and perform interlock control based on the status analysis results.

[0082] In addition, the electronic device can communicate with the equipment in the dry quenching process. The dry quenching process site is also equipped with image acquisition devices such as cameras to collect images of the equipment in the dry quenching process. The electronic device communicates with the image acquisition devices and acquires images. After analyzing the acquired images and obtaining the equipment status, it performs interlock control on the equipment in the dry quenching process based on the equipment status.

[0083] One possible implementation scheme, such as Figure 1 As shown in the embodiment of this application, an equipment control method includes: in response to meeting the equipment status detection conditions of the target stage of the dry quenching process, acquiring target images of each detection area under the target stage; wherein, each detection area includes: the area where a specified component of the target equipment to be focused on in the target stage is located (step S101); performing status detection on the specified component in the target image of each detection area to obtain the status detection result of the specified component (step S102); determining the status detection result of the target equipment in the target stage based on the obtained status detection result (step S103); and performing interlock control on the target equipment based on the status detection result of the target equipment (step S104). The equipment control method provided in this application, when meeting the equipment status detection conditions of the target stage of the dry quenching process, acquires target images of each detection area where a specified component of the target equipment to be focused on in the target stage is located; then, by performing status detection on the specified component in the target image of each detection area, the status detection result of the specified component is obtained, and the status detection result of the target equipment in the target stage is determined accordingly; thus, the status detection result of the target equipment includes the status detection result of the specified component in the target equipment, and based on the status detection result of the target equipment, more accurate interlock control of the equipment can be performed.

[0084] Regarding step S101, in this embodiment of the application, the dry quenching process is divided into multiple stages, such as: the stage where an electric locomotive pulls a coke can loaded with red-hot coke to the bottom of the hoisting shaft; the stage where the hoist hook suspends the coke can's lifting lug and lifts the coke can to the top of the tower; the stage where the hoist carries the coke can laterally; the stage where the hoist carries the coke can lower; the stage where red-hot coke is loaded into the coke can, etc. The target stage can be any of the above stages. The above stages are merely examples; the target stage can be any stage in the dry quenching process, such as: the stage where the coke can is checked to ensure it is in the correct position before the hoist hook suspends the coke can's lifting lug; the stage where the red-hot coke is conveyed by belt conveyor; the stage where heat exchange occurs inside the dry quenching oven, etc. This embodiment of the application does not limit this.

[0085] The equipment status detection condition for the target stage can be: detecting the equipment control signal for the target stage. For example, if the target stage is the stage where the hoist hook suspends the coke can lifting lug and lifts the coke can to the top of the tower, then the equipment control signal can be the hook opening and positioning signal; if the target stage is the stage where the hoist lowers the coke can, then the equipment control signal can be the control signal for lowering the coke can. Additionally, the equipment status detection condition for the target stage can also be a time-related condition. For example, if the execution time of the target stage is detected, then the equipment status detection condition for the target stage is considered satisfied. The above-mentioned equipment status detection conditions for the target stage are merely examples; any condition associated with the target stage can be used as the equipment status detection condition for the target stage, and this application does not limit this.

[0086] Each detection area in the target stage includes the area where the specified components of the target equipment to be monitored in the target stage are located. That is, in this embodiment of the application, the specified components are first detected in order to accurately confirm the equipment status of the target equipment. The equipment to be monitored in different stages may be the same or different. For example: if the target stage is the stage where the hoist hook suspends the coke can lifting lug and lifts the coke can to the top of the tower, then the target equipment to be monitored includes the hoist and the coke can; if the target stage is the stage where the hoist lowers the coke can, then the target equipment to be monitored includes the hoist, the coke can, and the hoisting shaft at the lowering position of the coke can; if the target stage is the stage where the coke can is rotated into place before the hoist hook suspends the coke can lifting lug, then the target equipment to be monitored includes the coke can; if the target stage is the stage where the belt conveys the red-hot coke, then the target equipment to be monitored includes the belt; if the target stage includes the stage of heat exchange in the dry quenching coke oven, then the target equipment includes the deaerator, the steam drum level gauge, etc. In addition, even if different stages involve the same equipment, the specified components to be monitored for the same equipment in different stages may be the same or different.

[0087] The state of a specific component within the target equipment affects the overall state of the target equipment. For example, if the target equipment includes a hoist and coke canisters, the specified components could include hooks, lifting lugs, and, of course, lifting ropes. If the target equipment includes a hoist, coke canisters, and a hoisting shaft, the specified components could include hooks, lifting lugs, the bottom gate of the coke canister, lifting ropes, and components in the hoist used to secure the coke canister (such as an APS (Auto Positioning System) device). Each detection area can correspond to one or more specified components. Acquiring the target image of each detection area means acquiring an image containing each specified component, allowing for state detection of each specified component.

[0088] In this embodiment of the application, for each stage of the dry quenching process, each detection area under each stage is equipped with an image acquisition device to acquire the target image of each detection area under each stage; when the equipment status detection conditions of the target stage of the dry quenching process are met, the target image of each detection area under the target stage can be obtained from the image acquisition device of the target stage.

[0089] In addition, the image acquisition device in the target stage is used to acquire images of the area where a specified component is located in the target device. The field of view of the image acquisition device includes the area where the specified component is located. For example, the image acquisition device is at the same horizontal height as the specified component and is directly facing the specified component. If there are multiple specified components, there are also multiple image acquisition devices, and the number of image acquisition devices is the same as the number of specified components. One image acquisition device corresponds to one specified component and is used to acquire images of the area where the corresponding specified component is located.

[0090] For step S102, for each detection area, there can be one or more target images; the method of performing state detection on a specified component in the target image will be described in detail in subsequent embodiments, and will not be repeated here.

[0091] The status detection results differ for different designated components. For example, if the designated component is a hook, the status detection results can be such as fully closed, not closed, loosely attached, partially attached, or fully closed; alternatively, they can be whether the hook has crossed the boundary or not. If the designated component is a bottom gate, the status detection results can be such as whether there is coke jamming, coke trapped, the bottom gate not fully closed, about to trap coke, or normal closure. If the designated component is a conveyor belt, the status detection results can be the temperature of the material on the belt, or the identification of abnormally high-temperature materials. If the designated component is a deaerator or a steam drum level gauge, the status detection results can be the liquid level height, scale position, and wave. The status detection results can be used to determine whether the deaerator or steam drum level gauge is abnormal (e.g., high or low level, blurred display, sudden level change, or normal status). If the specified component is a coke tank, the status detection result can be a judgment of whether the coke tank has rotated into place. If the specified component is a lifting device, the status detection result can be information such as the current position, attitude, and tilt angle of the lifting device, or an identification result of whether the lifting device is abnormal (e.g., tilt, offset, abnormal height difference on both sides, uneven force, or normal status). If the specified component is an automatic alignment and clamping device, the status detection result can be parameters such as clamping closing angle and fit, or an identification result of whether it is clamped. In this application, the focus of the detection of a specified component and its status at any target stage is related to that target stage. Specifically, the specified components to be detected and the status detection of the specified components can be set according to the target stage. The status detection of the same specified component at different target stages can be the same or different (e.g., before the coke can is hoisted by the hoist hook, the status detection is to check whether the coke can has rotated into place; when the coke can is hoisted by the hoist hook, the status detection is to check whether the hoist hook is closed with the hook and whether it has crossed the boundary). This application does not limit this.

[0092] In addition, before performing state detection on the target image, preprocessing such as denoising, enhancement, distortion correction, region cropping, illumination compensation, or edge enhancement can be performed on the target image to improve the image quality and ensure the accuracy of the state detection results.

[0093] Regarding step S103, after performing state detection on the specified components in the target image of each detection area, the detection results of each specified component are obtained. The state detection results of the target device can be understood as including the state detection results of each specified component. For example, the state detection results of each specified component can be directly used as the state detection results of the target device; or, based on whether each specified component is in a normal state as represented by the state detection results of each specified component, it can be determined whether the target device is in a normal state, and the state detection results of the target device can be obtained.

[0094] Specifically, if the status detection result of any component indicates that the specified component is not in a normal state, then the target device is determined to be in a normal state; if the status detection results of each specified component indicate that each specified component is in a normal state, then the target device is determined to be in a normal state.

[0095] Regarding step S104, the status detection result of the target equipment can indicate whether the target equipment is in a normal state. If the target equipment is in a normal state, then interlock control is applied to the target equipment, i.e., the target equipment continues to perform the actions of the target stage. For example, if the target equipment includes a hoist and a coke can, and the status detection result of the target equipment is that the hook and lifting lug are normally closed and the lifting rope is in a normal state, or if the status detection result of the target equipment is that the target equipment is in a normal state, then interlock control is applied to control the hoist to lift the coke can. If the target equipment is not in a normal state, then interlock control is applied to the target equipment, i.e., the target equipment is prohibited from performing the actions of the target stage, for example, directly controlling the target equipment to stop operating.

[0096] One possible implementation, such as Figure 2 As shown, state detection is performed on a specified component in the target image of each detection region to obtain the state detection result of the specified component. This includes: for each detection region's target image, obtaining state description information of the specified component in the target image (step S201); and determining the state detection result of the specified component based on the state description information (step S202). It can be seen that for each detection region's target image, state description information of the specified component can be obtained first; and the state description information can describe the state of the specified component in the target image, allowing for accurate determination of the state detection result of the specified component based on the state description information.

[0097] Regarding step S201, the state description information of the specified component can describe the state of the specified component in the target image. The state description information of the specified component may include: the geometric relationship of the specified component, the pose information of the specified component, and / or the classification result of the specified component, etc.

[0098] For example, the geometric relationship of a specified component can be understood as the geometric relationship between the specified component and other components. For instance, if the specified component is a hook, the geometric relationship of the specified component is the positional relationship between the hook and the lifting lug, such as: tightly fitted, not fitted, or adjacent on the left and right. The pose information of the specified component is the posture of the hook in the target image, such as: deflection angle, etc. The classification result of the specified component can be a state classification result with confidence level. For instance, if the specified component is the bottom gate of a coke tank, its classification result can include: closed or clamped, with corresponding confidence levels of 80% and 20%, respectively.

[0099] It should be noted that different status description information is obtained in different ways, and the specific methods will be described in detail in subsequent embodiments, and will not be repeated here.

[0100] Regarding step S202, the state description information can characterize the state of a specified component, thereby determining the state detection result of the specified component. For example, if the state description information is the geometric relationship of the specified component, which can be a hook or a bottom gate, the state of the specified component can be determined based on its geometric relationship. If the state description information is the pose information of the specified component, which can be a hook, the state of the hook can be determined based on its pose information. If the state description information is the state classification result, which can be a hook or a bottom gate, the state of the specified component can be determined based on its classification result. For example, the state classification result with the highest confidence level can be determined as the state detection result of the specified component.

[0101] Different state description information determines the state detection result of a specified component in different ways. The specific implementation methods will be described in detail in subsequent embodiments, and will not be repeated here.

[0102] One possible implementation, step S201 above includes: for each detection region's target image, determining the positions of multiple designated components in the target image; determining the target geometric relationship of the multiple designated components in the target image based on the determined positions; step S202 above includes: based on a pre-set first correspondence between the geometric relationship of the multiple designated components in the detection region and the component state, determining the target component state corresponding to the target geometric relationship, and obtaining a first state detection result of the designated component; and / or, step S201 above includes: for each detection region's target image, determining the target pose information of the designated component in the target image; step S202 above includes... The steps are as follows: S1: Based on the target pose information, determine the target contour of the specified component in the target image; determine whether the target contour is within the boundary range of the specified component in the pre-set detection area; if so, determine that the specified component has not crossed the boundary; otherwise, determine that the specified component has crossed the boundary, and obtain the second state detection result of the specified component; and / or, the above step S201 includes: for the target image of each detection area, determine the component detection area of ​​the specified component in the target image; classify the state of the specified component according to the component detection area, and obtain the state classification result of the specified component; the above step S202 includes: based on the state classification result of the component, determine the third state detection result of the specified component. It can be seen that the state description information of the specified component can be the target geometric relationship, target pose information, and / or state classification result of the specified component; when determining the state description information, it can be determined according to the different state description information and their corresponding methods; then, the state detection result of the specified component is determined according to the corresponding methods of different state description information. In this way, multiple methods for determining state description information and state detection results improve the richness of the determined state detection results, and the determination methods for determining state description information and state detection results can be flexibly selected according to actual needs.

[0103] For cases where the state description information is a geometric relationship, where the geometric relationship refers to the geometric relationship between multiple specified components in the target image, such as the geometric relationship between a hook and a lifting lug, then for each detection area of ​​the target image, the positions of the multiple specified components in the target image are first determined, and then the target geometric relationship of the multiple specified components is determined based on the determined positions. For example, first, the positions of the hook of the hoist and the lifting lug of the coke can are determined in the target image. Then, based on the positions of the hook and the lifting lug, the target geometric relationship between the hook and the lifting lug is determined. For example: if the position frames of the hook and the lifting lug overlap and their depth information is the same (or the depth difference between them is within the depth threshold range), then the target geometric relationship between the hook and the lifting lug is a close fit; if the position frames of the hook and the lifting lug overlap but their depth information is different, then the target geometric relationship between the hook and the lifting lug is a non-fitted front-back relationship; if the position frames of the hook and the lifting lug do not overlap, then the target geometric relationship between the hook and the lifting lug is left-right adjacent.

[0104] When the state description information is the target geometric relationship, a first correspondence between the geometric relationship of multiple specified components in the detection area and the component state can be preset. For example, the component state corresponding to the tightly fitted geometric relationship is the closed state, the component state corresponding to the non-fitted front-back relationship is the loose hanging state, and the component state corresponding to the left and right adjacent relationships is the unclosed state. Based on the first correspondence, the target component state corresponding to the target geometric relationship can be determined, and the target component state can be used as the first state detection result of the specified component.

[0105] It should be noted that the focus of the first state detection result is to detect whether the specified component is in a closed state or in other abnormal states, such as: loose engagement, off-center engagement, or not fully closed, all of which can be understood as abnormal states.

[0106] When the state description information is pose information, the pose information is the position and orientation of the specified part in the target image. The orientation information can be represented by the angle information. Therefore, for the target image of each detection area, the bounding box and rotation angle information of the specified part in the target image can be determined as the target pose information of the specified part.

[0107] When the state description information is pose information, the focus of state detection for a specified component is whether the component has exceeded its boundaries. For example, a hook should be suspended at the position of the lifting lug; if the hook is suspended in another position, it is considered to have exceeded its boundaries. During state detection, a pre-defined boundary range is set for the specified component. First, the target contour of the specified component in the target image can be determined based on the target pose information. Then, it is determined whether the target contour is within the boundary range. If it is, the specified component is determined not to have exceeded its boundaries; otherwise, the specified component is determined to have exceeded its boundaries. The state of exceeding or not exceeding the boundaries is used as the second state detection result for the specified component.

[0108] For example, the specified component is a hook, and the target pose information is the hook's position frame carrying angle information. First, based on the angle information, the coordinates of the center point of the hook's position frame, and the width and height of the position frame, the four corner points of the hook's position frame are determined. If the positions of all four corner points are within the boundary range, the hook has not exceeded the boundary; otherwise, it has exceeded the boundary. Here, the angle information is θ, the width and height of the position frame are w and h respectively, and the coordinates of the center point are (cx, cy). Therefore, the coordinates of the four corner points are as follows:

[0109] p1 = ( cx - dx×cosθ + dy×sinθ, cy - dx×sinθ - dy×cosθ );

[0110] p2 = ( cx + dx×cosθ + dy×sinθ, cy + dx×sinθ - dy×cosθ );

[0111] p3 = ( cx + dx×cosθ - dy×sinθ, cy + dx×sinθ + dy×cosθ );

[0112] p4 = ( cx - dx×cosθ - dy×sinθ, cy - dx×sinθ + dy×cosθ ); dx = w / 2, dy = h / 2.

[0113] For cases where the state description information is a state classification result, for each target image of a detection region, first determine the component detection region of the specified component in the target image, and then classify the state of the component detection region. For example, call the state classifier corresponding to the specified component to classify the state of the specified component. Different specified components correspond to different state classifiers, and any state classifier corresponding to a specified component can be trained using images of various states of that specified component. For example, if the specified component is a hoist hook, its corresponding classifier can be trained using images of the hook in closed, open, partially hooked, or off-center states; if the specified component is a coke oven bottom gate, its corresponding classifier can be trained using images of the bottom gate in states such as coke jamming, coke clamping, incomplete closure, or about to clamp.

[0114] When the state description information is a state classification result, the state classification result can directly characterize the confidence level of a specified component belonging to a certain state. Therefore, the state with the highest confidence level can be determined as the third state detection result of the specified component.

[0115] It should be noted that the first state detection result mentioned above focuses on detecting the state of a specified component through geometric relationships, the second state detection result focuses on detecting whether the specified component has crossed boundaries through pose information, and the third state detection result focuses on detecting the state of a specified component through state classification. Furthermore, one or more methods can be used to detect the state of a specified component. If only one method is used, the state detection result of the specified component is directly obtained; if multiple methods are used, the state detection results of the multiple methods can be fused to obtain the state detection result of the specified component. The specific implementation method will be described in detail in subsequent embodiments and will not be elaborated here.

[0116] Furthermore, for any target image in any detection area under the target stage, the specified components can be state-detected in the manner described above as required to obtain state detection results. For example, in the stage where the hoist hook suspends the coke can lifting lug and carries the coke can to the top of the tower, the specified components include the hook, the rope, and the lug (the lug is adjacent to the hook). The state of whether the hook is closed can be detected according to the geometric relationship, and the state of whether the hook has crossed the boundary can be detected according to the pose information. Of course, the state classification results can also be used to detect whether the hook is closed, etc. This application embodiment does not limit this.

[0117] In one possible implementation, the target image of any detection region includes: a series of consecutive frames of images acquired for the detection region; performing state detection on a specified component in the target image of each detection region to obtain a state detection result for the specified component, including: performing state detection on a specified component in a series of consecutive frames of images for each detection region to obtain a detection result for each of the series of consecutive frames; determining a state detection result for the specified component based on the detection results for each of the series of consecutive frames; wherein, the method of performing state detection on a specified component in any frame image includes: determining a specified state detection result for the specified component in that frame image; wherein the specified state detection result includes at least one of the following: a first state detection result, a second state detection result, and a third state detection result; determining a state detection result for the specified component based on the specified state detection result. It is evident that the target image of any detection region can be a series of consecutive frames of images acquired for the detection region. Therefore, when performing state detection, state detection can be performed on a specified component in a series of consecutive frames of images for each detection region, and the state detection result for the specified component can be determined using the detection results for each of the series of consecutive frames. Compared to detecting from a single image, this method can improve the accuracy of the state detection result. Furthermore, when performing state detection on a specified component in any frame of an image, one or more methods can be used to determine the specified state detection result of the specified component in that frame of the image. Then, based on at least one state detection result included in the specified state detection result, the state detection result of the specified component can be determined. Compared with the method of determining the state detection result of the specified component in a single way, the accuracy of the state detection result of the specified component can be improved.

[0118] When performing state detection on a specified component in any frame of an image, one or more of the aforementioned geometric relationships, pose information, and state classification methods can be used. Therefore, for a specified component in that frame, at least one of a first state detection result, a second state detection result, and a third state detection result can be obtained. Then, based on the specified state detection result, the state detection result of the specified component is determined. For example, if the specified state detection result includes multiple state detection results, a selection priority can be set for the multiple state detection results, or a weight can be set for each state detection result to fuse the multiple state detection results, etc., to obtain the state detection result of the specified component. For instance, if state detection is performed using geometric relationships and state classification methods, and the state detection results of the two methods are the same, then the same detection result is used as the state detection result of the specified component. If they are different, and the priority of the first state detection result is greater than the priority of the third state detection result, then the first state detection result is used as the state detection result of the specified component in that frame of the image.

[0119] Furthermore, when determining the state detection result of a specified component based on the detection results of multiple consecutive frames, the state detection results of the specified component in multiple consecutive frames can be fused (e.g., weighted and fused according to each frame's state detection result), or, according to priority, the state detection result of the specified component in a single frame (e.g., the last frame) can be selected as the final state detection result of the specified component. The above methods for determining the state detection result of a specified component based on the detection results of multiple consecutive frames are merely examples; any method capable of determining the state detection result of a specified component based on the detection results of multiple consecutive frames is applicable to this application.

[0120] In one possible implementation, for each detection region's target image, obtaining state description information of a specified component in the target image includes: for each detection region's target image, calling the state detection model corresponding to that detection region to perform state detection on the specified component in the target image, thereby obtaining the state description information of the specified component; wherein, as... Figure 3 As shown, the training method for the state detection model corresponding to any detection region includes: acquiring multiple initial sample images of the detection region; wherein each initial sample image contains a specified component in the detection region (step S301); calling the state detection model to be trained for the detection region to perform state detection on the multiple initial sample images, obtaining the initial state detection results of each initial sample image (step S302); determining the state calibration ground truth value of each initial sample image based on the initial state detection results of each initial sample image (step S303); training the state detection model to be trained based on the state calibration ground truth value of each initial sample image, to obtain the trained state detection model corresponding to the detection region (step S304). It can be seen that in this embodiment, a corresponding state detection model is set for each detection region to adapt to the state detection of specified components in the target image of each detection region, thereby improving the accuracy of the state description information of specified components in the target image of each detection region. During training, the state detection model corresponding to any detection region first calls the state detection model to be trained to perform state detection on multiple initial sample images of the detection region. Based on the obtained detection results, the state calibration ground value of each initial sample image is determined, which can reduce the resource consumption required for calibrating the state of each initial sample image during the training process of the state detection model. Then, the state calibration ground value of each initial sample image is used to train the state detection model, which can improve the training efficiency of the state detection model.

[0121] In this embodiment of the application, a corresponding state detection model is set for different detection areas. The state detection model corresponding to each detection area is used to perform state detection on a specified component under that detection area to obtain the state description information of the specified component. In addition, the state detection model corresponding to each detection area can also use the state description information of the specified component to determine the state detection result of the specified component. This embodiment of the application does not limit this.

[0122] It should be noted that when the state detection model corresponding to each detection area performs state detection on a specified component, it can flexibly select geometric relationships, pose information and / or state classification to obtain the state description information of the specified component. This application embodiment does not limit this.

[0123] For step S301, for any detection area, each initial sample image contains a specified component in the detection area. The initial sample image can be an unlabeled image. Multiple initial sample images of the detection area can be obtained by querying from the database, or the image of the detection area can be acquired in real time as the initial sample image. This application embodiment does not limit this.

[0124] For step S302, the state detection model to be trained has initial model parameters and is able to perform state detection on the specified component; then the state detection model to be trained in the detection area can be called first to perform preliminary calibration on multiple initial sample images and obtain the initial state detection results of each initial sample image.

[0125] For step S303, since the state detection model has not yet been trained or has not been fully trained, the state in the initial state detection results of each initial sample image is not completely accurate. Therefore, it is necessary to further determine the state calibration truth value of each initial sample image. For example, a manual review method can be used to determine whether the state represented in the initial state detection result of each initial sample image is accurate. If it is accurate, it can be directly used as the state calibration truth value of the initial sample image. If it is inaccurate, the state calibration truth value of the initial sample image can be obtained manually.

[0126] For step S304, after obtaining the state calibration ground truth of each initial sample image, the state detection model to be trained can be trained. During the training process, the model parameters of the state detection model can be adjusted so that the state detection result of the state detection model for the initial sample image after parameter adjustment is close to the state calibration ground truth of the initial sample image, until the model converges. For example, if the accuracy of the state detection model for the state recognition result of the specified component in the detection area reaches 90%, then the state detection model corresponding to the detection area after training is obtained.

[0127] In one possible implementation, the method further includes: for any detection region corresponding to a state detection model, determining the state detection ground truth of a specified component based on the state description information of the specified component obtained by the state detection model; and updating and training the state detection model based on the state detection ground truth. Therefore, the state detection model corresponding to any detection region can also utilize the state description information of the specified component obtained by the state detection model to determine its state detection ground truth, and use the state detection ground truth to update and train the state detection model, thereby improving the accuracy of the state detection model.

[0128] The trained state detection model may have insufficient accuracy in subsequent applications. Therefore, for any detection area, the state detection model can be updated by determining the state detection ground truth of the specified component based on the state description information of the specified component obtained by the state detection model during the application process. The updated training method is similar to the original training method and will not be elaborated here.

[0129] In addition, the state detection model can directly obtain the state detection results of a specified component, and can also determine the state detection truth value of a specified component based on the state detection results of the specified component. For example, the accuracy of the state detection result can be judged by manual review. If it is accurate, it can be directly used as the state detection truth value. If it is inaccurate, it can be manually recalibrated to obtain the state detection truth value.

[0130] In one possible implementation, interlocking control of the target equipment is performed based on the status detection result of the target equipment. This includes: if the status detection result indicates that the target equipment is in a normal state, then an interlocking control signal is sent to the target equipment to control it to perform the action of the target stage; otherwise, a prohibition signal is sent to the target equipment to prohibit its action. Therefore, by automatically sending corresponding signals to the target equipment when the status detection result indicates that it is in a normal or abnormal state, the system can automatically control it to perform the action of the target stage or prohibit the action, thereby improving the automation level of the dry quenching process.

[0131] If the status detection result of the target equipment indicates that the target equipment is in a normal state, an interlock control signal is sent to the target equipment in the target stage. For example, if the target stage is the stage where the hoist hook suspends the coke can lifting lug and lifts the coke can to the top of the tower, an interlock control signal is sent to the hoist's PLC to enable the hoist to control the hook to lift the coke can. If it is the stage of loading red coke into the coke can, control signals are sent to the locomotive, coke quenching car, and coke pushing car to load red coke into the coke can. Otherwise, if the status detection result of the target equipment indicates that the target equipment is in an abnormal state (i.e., at least one specified component is in an abnormal state), considering the on-site safety of the dry quenching process, a prohibition signal is sent to the target equipment to prohibit its operation until the target equipment is subsequently identified as being in a normal state and the operation of the target stage continues. After sending the prohibition signal, an alarm can also be generated, such as through voice or sound and light warnings.

[0132] In addition, in practical applications, data such as target images, status detection results of specified components, status detection results of target devices, alarm events, processing results of alarm events, and timestamps corresponding to the above can be displayed in real time and written into the database for storage, so as to facilitate information display, information traceability analysis, and subsequent optimization and operation statistics.

[0133] Based on the above method embodiments, this application also provides a device control system, such as... Figure 4 As shown, the system includes: an image acquisition unit 410, a recognition unit 420, and a control unit 430;

[0134] The image acquisition unit 410 is configured to acquire target images of each detection area during the target stage; wherein each detection area includes: the area where a specified component is located in the target device that needs to be focused on during the target stage;

[0135] The control unit 430 is configured to, in response to the equipment status detection conditions that meet the target stage of the dry quenching process, acquire target images of each detection area under the target stage from the image acquisition unit 410; and send the target images of each detection area under the target stage to the recognition unit 420.

[0136] The identification unit 420 is configured to receive target images of each detection area under the target stage, perform state detection on a specified component in the target image of each detection area, obtain the state detection result of the specified component, and feed back the state detection result of the specified component to the control unit 430.

[0137] The control unit 430 is also configured to receive the status detection result of a designated component, determine the status detection result of the target device at the target stage based on the obtained status detection result, and perform interlocking control on the target device based on the status detection result of the target device.

[0138] In one possible implementation, the system further includes: an interlocking control interface module; the interlocking control interface module is provided with interfaces that need to be called when performing interlocking control at each stage;

[0139] The control unit performs interlocking control on the target device based on the status detection results of the target device, including:

[0140] If the status detection result of the target device indicates that the target device is in a normal state, the target interlocking control interface corresponding to the target stage in the interlocking control interface unit is called to send an interlocking control signal to the target device to control the target device to perform the action of the target stage; otherwise, the prohibition interface corresponding to the target stage in the interlocking control interface unit is called to send a prohibition signal to the target device to prohibit the action of the target device.

[0141] In one possible implementation, the system further includes a lighting unit, an alarm unit, a display unit, and a data storage unit;

[0142] The illumination unit is configured to provide a light source when the image acquisition unit acquires target images of each detection area during the target phase.

[0143] The alarm unit is configured to issue an alarm signal when the status detection result of the target device indicates that the target device is not in a normal state;

[0144] The display unit is configured to acquire target images of each detection area under the target stage from the image acquisition unit, acquire the status detection results of the specified component from the recognition unit, acquire the alarm events represented by the alarm signals from the alarm unit, acquire the status detection results of the target device from the control unit, and display the acquired content.

[0145] The storage unit is configured to acquire target images of each detection area in the target stage from the image acquisition unit, acquire the status detection results of a specified component from the recognition unit, acquire the alarm events represented by alarm signals from the alarm unit, acquire the status detection results of the target device from the control unit, and store the acquired content.

[0146] The following describes a device control method and system provided in this application with reference to a specific embodiment.

[0147] The purpose of this application is to provide an equipment control method and system to solve the problems in the prior art, such as inaccurate identification of key states such as hook closure, hook over-boundary, bottom gate clamping and closure, insufficient automatic interlocking capability, poor adaptability to complex environments, and lack of full-process data closed loop.

[0148] The device control system provided in this application includes: an image acquisition unit, a lighting unit, a recognition unit, an interlocking control interface module, a control unit, a display unit, a data storage unit, and an alarm unit.

[0149] Image acquisition unit: Composed of one or more high-definition industrial cameras or video cameras, used to acquire real-time images or video streams of the left and right hooks of the hoist, the coke pot lifting lug area, the APS device of the hoisting shaft, and the bottom gate area. It can also acquire images or video streams of the detection areas of other designated components, without limitation. Alternatively, the image acquisition unit can also employ industrial cameras, network cameras, or thermal / visible light dual-light cameras, etc.

[0150] Lighting unit: includes fill light, mounting bracket, protective structure and positioning mechanism, used to improve image quality in night, backlight, rain, fog and dust environments; positioning mechanism is used to ensure that the fill light and other equipment can be put back in their original position after replacement, ensuring the ability to provide light source for image acquisition unit.

[0151] Recognition Unit: Deploys a visual AI controller, industrial PC, or GPU computing node (or server). The industrial PC is used to preprocess the acquired target image, and the AI ​​controller or GPU computing node performs target detection, status recognition, boundary judgment, and abnormal status determination on the specified parts in the target image.

[0152] Interlocking Control Interface Unit: This unit includes interlocking control interfaces for various target stages. It transmits control signals obtained from the status detection results of the target equipment to the hoist PLC, locomotive, or other actuators via MQTT (Message Queuing Telemetry Transport), HTTP (Hypertext Transfer Protocol), REST (Representational State Transfer), OPC UA (Open Platform Communications Unified Architecture), or digital / network I / O methods to form interlocks. In other words, the control signals can be implemented via industrial Ethernet protocol or hard-contact I / O.

[0153] The control unit is used to control the image acquisition unit, recognition unit, interlocking control interface unit, etc., to realize equipment status recognition and equipment interlocking control in the dry quenching process; and to send the generated data to the display unit and storage unit, and control the alarm unit to issue alarms.

[0154] Display unit: Used to display real-time image acquisition footage, status detection results, alarm events, historical records, device status, and statistical reports. The control unit can be deployed within the display unit, allowing operators to control the device via its screen.

[0155] Data storage unit: Used to store target images, state detection results, logs, alarm records, labeled data and model versions of state detection models, and supports online and automatic labeling of training data (such as labeling through the model), and supports visual training and one-click deployment of state detection models;

[0156] Alarm unit: When an abnormal state is detected, it triggers alarms such as voice broadcast and sound and light prompts.

[0157] The identification unit may include:

[0158] Hook closure detection module: Based on target localization and deep learning recognition algorithms, it detects the area of ​​the coke can lifting lug and determines whether the hook is in a closed, unclosed, loosely attached, off-center, or other abnormal state; corresponding to the above-mentioned relationship between the hook and the lifting lug, it identifies the hook state and obtains the first state detection result of the hook.

[0159] Hook boundary crossing detection module: Based on the rotating target detection algorithm, the edge and relative posture of the lifting lug are identified, and a rotating rectangle with angle information is output. The boundary crossing is judged in combination with the set boundary range. Corresponding to the above judgment of whether the hook has crossed the boundary by using pose information, the second state detection result of the hook is obtained.

[0160] Bottom gate coke jamming and closure identification module: By identifying the target image at the bottom of the hoist or the bottom gate area, it determines whether there is a risk of coke jamming, coke jamming, incomplete closure, or imminent coke jamming. Corresponding to the above-mentioned state classification method, the bottom gate (which can also identify the bottom of the hoist) is classified into states to obtain the third state detection result of the bottom gate.

[0161] The coke can rotation positioning detection module analyzes video and image data in real time to accurately capture the dynamic movements of the coke can. Combining production and maintenance experience, it automatically triggers detection during the stage of checking whether the coke can has rotated into position, intelligently verifying the rotation status and efficiently determining whether the coke can has rotated to the correct position. If an abnormality is detected, an alarm mechanism is immediately triggered, quickly alerting production personnel to intervene, providing dual protection for the safe and stable progress of the dry quenching coke production process.

[0162] The belt conveyor red-coke detection module intelligently analyzes video and image data from the belt conveyor area using AI image recognition algorithms, focusing on identifying the condition of materials on the belt. It makes comprehensive judgments based on the material's color characteristics, brightness changes, area, and duration, promptly detecting the risk of incompletely extinguished coke entering the belt conveyor. If high-temperature materials, such as red-coke or abnormally high-temperature coke, are detected, an alarm is immediately triggered, prompting on-site personnel to take rapid action to prevent belt burns, fire hazards, and production interruptions, ensuring the safe and stable operation of the coke conveying system.

[0163] The intelligent identification and detection module for deaerator and steam drum level gauges acquires real-time video and image data from instruments such as deaerators and steam drum level gauges. Combining AI image and character recognition algorithms, it intelligently analyzes the display status of these gauges, automatically identifying liquid level height, scale position, and abnormal fluctuations. It can judge abnormal states such as excessively high or low liquid levels, blurred displays, and sudden changes in liquid level, and integrates this with production operation experience and set thresholds to achieve intelligent verification of key water level parameters. If an abnormal liquid level or suspected instrument display anomaly is detected, an alarm is triggered promptly, assisting operators in quickly confirming and handling the situation, reducing the possibility of equipment operation risks caused by abnormal liquid levels, and improving the operational safety of the boiler and water-steam system.

[0164] The spreader balance detection module acquires real-time image data of the spreader, coke can, and connecting parts. Combined with AI image analysis algorithms, it intelligently identifies the spreader's operating posture, focusing on determining whether there is tilting, offset, abnormal height differences on both sides, uneven stress, or other issues. Based on changes in spreader posture and the coke can's lifting status, it can monitor the balance and assess risks during the lifting process in real time. If any abnormalities such as spreader imbalance are detected, an alarm is immediately triggered, assisting operators in timely intervention and improving the safety and stability of coke can lifting operations.

[0165] APS Clamping Recognition Module: A high-definition industrial camera is deployed in the APS clamping area to capture real-time contact images of the clamping mechanism. The module combines AI algorithms to perform feature analysis on the images, accurately identifying key parameters such as clamping closure angle and fit, and determining whether the safe clamping threshold has been reached. APS is resistant to dust and high-temperature interference, and can replace traditional sensor detection, avoiding misjudgments due to mechanical wear. When abnormalities such as non-clamping or misalignment are detected, an audible alarm is immediately triggered, and the APS is shut down to ensure the safety of equipment and personnel in the dry quenching lifting and transfer process. Simultaneously, data logs are generated to assist in section operation and maintenance management.

[0166] Comprehensive State Judgment Module: For each detection area under any target stage, it combines the target image of continuous time frames, the first state detection result obtained from geometric relationships, the second state detection result obtained from pose information, and the third state detection result containing confidence level, etc., to generate the state detection result of the target device that can be used for interlocking.

[0167] Among them, the modules other than the comprehensive status judgment module can be flexibly selected according to different target stages to perform adaptive status detection on the specified components under the target stage, so as to achieve accurate identification of the equipment status under the target stage and to perform more accurate interlocking control of the equipment.

[0168] Based on the above equipment control system, the equipment control method includes the following steps:

[0169] S1. Cameras are installed at the hoist hook, coke tank lifting lugs, coke tank bottom gate and related key work positions to collect real-time video streams;

[0170] S2. When the equipment status detection conditions of the target stage are reached, acquire the target images of each detection area under the target stage, and perform preprocessing such as denoising, enhancement, distortion correction, region cropping, illumination compensation or edge enhancement on the target images.

[0171] S3. For any target image in any detection area, call the corresponding state detection model to perform state detection on the specified component and obtain state description information; when the target equipment includes hooks and lugs, output the position of hooks and lugs and the target geometric relationship; and output the position, angle and other information of hooks; when the target equipment includes coke cans, output the position of the bottom gate of the coke can, the coke edge and the state classification result.

[0172] S4. Based on the target geometric relationship, boundary range, and first correspondence, obtain the state detection result of the specified component under each frame image, and combine multiple frames of images to obtain the final fusion result, accurately determine the state of the specified component and the state of the target equipment; if the target equipment includes a hook, determine whether the hook is in a closed state or in an out-of-bounds state, etc.; if the target equipment includes a bottom gate, determine whether the bottom gate is in a clamped or closed state, etc.

[0173] S5. When the target equipment is determined to be in a normal state, an interlock control signal that allows interlocking is sent to the target equipment. For example, if the target stage is the stage where the hoist hook suspends the coke can lifting lug and lifts the coke can to the top of the tower, an interlock control signal is sent to the hoist PLC to enable the hoist to lift the coke can. When the determination result is abnormal or uncertain, a prohibition action signal is sent to the hoist PLC and an alarm is triggered simultaneously.

[0174] S6. In the above process, the target image, the status detection results of the specified component, the status detection results of the target device, alarm events, the processing results of the alarm events, and the timestamps corresponding to the above content are used for traceability analysis, status detection model optimization, and operation statistics; and the above data are displayed in real time.

[0175] S7. During operation, the model parameters are continuously updated using online annotation and incremental training mechanisms (i.e., using the detection results obtained from the state detection model to determine the true value of state detection and updating the state detection model) to adapt to changes in on-site working conditions.

[0176] In one possible implementation, the detection method for the hook closure state is as follows:

[0177] High-definition industrial cameras are installed in the corresponding areas of the hoist hook and the coke oven lifting lug. These cameras are mounted on fixed brackets to ensure stable observation of the hook-lifting lug engagement. After the target image is input into the recognition unit, it undergoes preprocessing such as noise reduction and brightness compensation. Then, the hook closure detection model is used to identify the hook's opening and closing contour, the lug position, and the relative contact relationship between the hook and lug. Based on the relative envelope relationship between the hook end and the lug, the closure angle, the contact integrity, and the stability across multiple frames, the system outputs a state detection result indicating whether the hook is "closed in place" or "not closed / abnormal." When the state detection result is abnormal, the hoist is immediately prohibited from performing the target stage's actions.

[0178] In one possible implementation, the detection method for the hook crossing the boundary is as follows:

[0179] Cameras and supplementary lighting devices are deployed at key observation directions where the lifting lugs may shift (such as the area near the lifting lugs in the coke oven). After preprocessing, the target image is input into a rotating target detection model. The model outputs the hook position and angle information, then determines the hook's outline and compares it with a pre-set hook boundary range. When the hook's posture exceeds the boundary range, it is considered an out-of-bounds error. To suppress false alarms caused by occasional jitter, a multi-frame confirmation mechanism can be used before outputting the final interlocking control signal.

[0180] In one possible implementation, the method for identifying the bottom gate's clamping and closing states is as follows:

[0181] Cameras are installed at the bottom of the hoist frame or at the bottom gate to capture target images between the bottom of the coke tank and the platform / gate. A classification model identifies the coke accumulation outline, gate edge position, and changes in the bottom gate gap to determine if there are any abnormal trends such as coke jamming, sticking, incomplete closure, or impending coke jamming. When an anomaly is detected, a pop-up window, audible and visual alarms, and voice announcements are triggered, and a prohibition signal is sent to the target equipment.

[0182] The aforementioned detection methods for the closed state of the hook and the over-boundary state of the hook can be applied to the stage when the hoist hook suspends the coke can's lifting lug and carries the coke can to the top of the tower; the aforementioned identification methods for the bottom gate clamping coke and its closed state can be applied to the stage when the hoist carries the coke can down or when red coke is loaded into the coke can, etc.

[0183] In addition, the front end of the equipment control system (such as the display unit) can display real-time images and equipment status of the left hook, right hook, APS device, bottom clamp, etc.; the back end (such as the data storage unit) provides functions such as alarm filtering, equipment management, view configuration, log tracing, online manual annotation, online automatic annotation, visual training of the status detection model, and one-click deployment of the status model. Through continuous retraining of false alarm and missed alarm samples, the adaptability of the status model in different factory areas and different seasons can be improved.

[0184] The equipment control system follows the safety principle of "better to refuse to operate than to release the operation by mistake" during operation. That is, it is permissible to identify an actual normal state as an abnormal state, resulting in temporary inability to interlock the action, but it is not permissible to identify an actual abnormal state as a normal state and release the action.

[0185] When the status recognition result meets the preset safety conditions (i.e., all specified components are in normal condition), an allow signal is output to the target equipment, such as the hoist PLC. When the recognition result does not meet the preset safety conditions, the target is incomplete, the recognition fails, or the confidence level is lower than the threshold, a prohibit signal is output to the target equipment, and an abnormal alarm is triggered. In other words, if the status of any specified component in the target stage is abnormal, the interlocking action cannot be performed; the interlocking action can only be performed when the status of all specified components in the target stage is normal. Furthermore, safety can be improved by accurately setting the first correspondence and reducing the boundary range.

[0186] The state control method and system provided in this application, through the combination of machine vision and deep learning, can achieve stable recognition under daytime, nighttime, different lighting conditions, and complex weather conditions, improving the detection reliability of key working conditions; by directly sending the recognition results to the target device and participating in interlocking control, it achieves an upgrade from "manual judgment" to "automatic protection"; it adopts a rotating target detection and multi-frame fusion mechanism to improve the recognition capability of posture changes, occlusion interference, and boundary anomalies; the system integrates real-time monitoring, alarm management, device management, view management, annotation training, and log traceability to form a complete data closed loop; it is compatible with newly installed industrial cameras and old cameras, supports standard industrial communication interfaces, and is easy to promote and implement in dry quenching sites; based on the on-site project objectives, the system can achieve an anomaly recall rate of no less than 99.5%, a system operating rate of no less than 99.5%, and the interlocking signal response time can be controlled within 2 seconds.

[0187] In one possible implementation, such as Figure 5 As shown, the above-mentioned equipment control system can be divided into a user access layer, a business application layer, a platform capability layer, a model management layer, and an infrastructure and interface layer.

[0188] The user access layer contains a platform with a control unit deployed for administrators, maintenance personnel, team leaders, operators, or third-party systems to access and use.

[0189] The business application layer includes functions such as real-time monitoring, task view, alarm management, alarm handling, device management, view management, identification record tracing, data statistics, security logs, and voice broadcast.

[0190] The platform's capability layer supports device (such as camera) access management, video stream management, rule engine, recognition service, alarm engine, data analysis service, reporting service, user and permission management, log auditing, and message notification service.

[0191] The model management layer supports data acquisition, online annotation, automatic annotation, model training, model evaluation, model deployment, and algorithm package management. The algorithm package management includes algorithms for detecting hook closure status, detecting hook boundary crossing status, and recognizing bottom gate clamping and closure status.

[0192] The infrastructure and interface layer includes: local area network, API interface, MQTT, HTTP, REST, OPC UA, etc. of interlocking control interface unit, camera of image acquisition unit, visual AI controller and GPU server of recognition unit, database server and storage of data storage unit, and other execution equipment such as hoist PLC and locomotive.

[0193] The device control system provided in this application provides centralized management of devices, data, models, and services, reducing the operation and maintenance costs of decentralized management; standardized model components and templates are ready to use out of the box, enabling rapid online deployment of models; data-driven closed-loop optimization can continuously improve the accuracy and scenario adaptability of the state detection model; and full-link logging and auditing ensure operation traceability, security and compliance, and traceability and callback capabilities.

[0194] This application also provides an electronic device, such as... Figure 6 As shown, it includes a processor 601, a communication interface 602, a memory 603, and a communication bus 604, wherein the processor 601, the communication interface 602, and the memory 603 communicate with each other through the communication bus 604.

[0195] Memory 603 is used to store computer programs;

[0196] The processor 601, when executing the program stored in the memory 603, implements any of the described device control methods.

[0197] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0198] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0199] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0200] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0201] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of any of the above-described device control methods.

[0202] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the device control methods described above.

[0203] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0204] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0205] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0206] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. A device control method, characterized in that, The method includes: In response to the equipment status detection conditions for meeting the target stage of the dry quenching process, target images of each detection area under the target stage are acquired; wherein, each detection area includes: the area where a specified component is located in the target equipment that needs to be focused on in the target stage; Perform state detection on specified components in the target image of each detection area to obtain the state detection results of the specified components; Based on the obtained status detection results, determine the status detection results of the target device in the target stage; Based on the status detection results of the target device, interlock control is performed on the target device.

2. The method according to claim 1, characterized in that, The process of performing state detection on specified components in the target image of each detection region to obtain the state detection results of the specified components includes: For each detection area, the state description information of a specified component in the target image is obtained. Based on the state description information, determine the state detection result of the specified component.

3. The method according to claim 2, characterized in that, For each detection region, the state description information of a specified component in the target image is obtained, including: For each detection area of ​​the target image, the positions of multiple specified components in the target image are determined; Based on the determined locations, determine the target geometric relationships of multiple specified components in the target image; Determining the state detection result of the specified component based on the state description information includes: Based on the first correspondence between the geometric relationship and the state of multiple specified components in the detection area, the state of the target component corresponding to the target geometric relationship is determined, and the first state detection result of the specified component is obtained. And / or, for each detection region, obtaining state description information of a specified component in the target image includes: For each detection region's target image, determine the target pose information of a specified component in the target image; Determining the state detection result of the specified component based on the state description information includes: Based on the target pose information, determine the target contour of the specified component in the target image; Determine whether the target contour is within the boundary range of a designated component in the pre-set detection area. If yes, determine that the designated component has not crossed the boundary; otherwise, determine that the designated component has crossed the boundary, and obtain the second state detection result of the designated component. And / or, for each detection region, obtaining state description information of a specified component in the target image includes: For each detection region of the target image, determine the component detection region of the specified component in the target image; The state of the specified component is classified according to the component detection area to obtain the state classification result of the specified component. Determining the state detection result of the specified component based on the state description information includes: Based on the state classification result of the component, determine the third state detection result of the specified component.

4. The method according to claim 3, characterized in that, The target image of any detection region includes: multiple consecutive frames of images acquired for that detection region; the step of performing state detection on a specified component in the target image of each detection region to obtain the state detection result of the specified component includes: Perform state detection on a specified component in multiple consecutive frames of images for each detection region to obtain the detection results for each of the multiple consecutive frames. Based on the detection results of each of the consecutive frames of images, determine the state detection result of the specified component; The methods for performing state detection on a specified component in any frame of an image include: Determine the detection result of a specified state of a specified component in the frame image; wherein the specified state detection result includes at least one of the following: the first state detection result, the second state detection result, and the third state detection result; Based on the specified state detection result, the state detection result of the specified component is determined.

5. The method according to claim 2, characterized in that, For each detection region, the state description information of a specified component in the target image is obtained, including: For each detection region of the target image, the state detection model corresponding to that detection region is called to perform state detection on the specified component in the target image and obtain the state description information of the specified component. The training methods for the state detection model corresponding to any detection region include: Acquire multiple initial sample images of the detection area; wherein each initial sample image contains a specified component in the detection area; The state detection model to be trained in the detection region is invoked to perform state detection on multiple initial sample images, and the initial state detection results of each initial sample image are obtained. Based on the initial state detection results of each initial sample image, determine the state calibration true value of each initial sample image; Based on the state calibration ground values ​​of each initial sample image, the state detection model to be trained is trained to obtain the state detection model corresponding to the detection region after training.

6. The method according to claim 5, characterized in that, The method further includes: For any detection area corresponding to the state detection model, the state detection true value of the specified component is determined based on the state description information of the specified component obtained from the state detection model. The state detection model is updated and trained based on the state detection ground truth.

7. The method according to claim 1, characterized in that, The step of interlocking control of the target device based on the status detection result of the target device includes: If the status detection result of the target device indicates that the target device is in a normal state, an interlock control signal is sent to the target device to control the target device to perform the action of the target stage; otherwise, a prohibition signal is sent to the target device to prohibit the action of the target device.

8. A device control system, characterized in that, The system includes: an image acquisition unit, a recognition unit, and a control unit; The image acquisition unit is configured to acquire target images of each detection area during the target stage; wherein, each detection area includes: the area where a specified component is located in the target device that needs to be focused on during the target stage; The control unit is configured to, in response to the equipment status detection conditions of the target stage of the dry quenching process, acquire target images of each detection area under the target stage from the image acquisition unit; and send the target images of each detection area under the target stage to the recognition unit. The identification unit is configured to receive target images of each detection area under the target stage, perform state detection on a specified component in the target image of each detection area, obtain the state detection result of the specified component, and feed back the state detection result of the specified component to the control unit. The control unit is further configured to receive the status detection result of a designated component, determine the status detection result of the target device in the target stage based on the obtained status detection result, and perform interlocking control on the target device based on the status detection result of the target device.

9. The system according to claim 8, characterized in that, The system also includes: an interlocking control interface module; the interlocking control interface module is equipped with interfaces that need to be called when performing interlocking control at each stage; The control unit performs interlocking control on the target device based on the status detection result of the target device, including: If the status detection result of the target device indicates that the target device is in a normal state, then the target interlocking control interface corresponding to the target stage in the interlocking control interface unit is invoked to send an interlocking control signal to the target device to control the target device to perform the action of the target stage; otherwise, the prohibition interface corresponding to the target stage in the interlocking control interface unit is invoked to send a prohibition signal to the target device to prohibit the action of the target device.

10. The system according to claim 9, characterized in that, The system also includes a lighting unit, an alarm unit, a display unit, and a data storage unit; The illumination unit is configured to provide a light source when the image acquisition unit acquires target images of each detection area during the target stage. The alarm unit is configured to issue an alarm signal when the status detection result of the target device indicates that the target device is not in a normal state; The display unit is configured to acquire target images of each detection area under the target stage from the image acquisition unit, acquire the status detection results of a specified component from the recognition unit, acquire the alarm event represented by the alarm signal from the alarm unit, acquire the status detection results of the target device from the control unit, and display the acquired content. The storage unit is configured to acquire target images of each detection area under the target stage from the image acquisition unit, acquire the status detection results of a specified component from the recognition unit, acquire the alarm event represented by the alarm signal from the alarm unit, acquire the status detection results of the target device from the control unit, and store the acquired content.

Images