Operation state determination method and device, electronic equipment and medium

Abstract

The embodiment of the invention provides an operation state determination method and device, electronic equipment and a medium. The method comprises the steps that after a tank body is controlled to move to a target position, a first image is collected through a collecting device, and the collecting end of the collecting device right faces the first side face of a lifting lug when the tank body is in an in-place state; based on the first image, offset data of the tank body in the X-axis direction is determined, and the X axis is perpendicular to the collecting end of the collecting device and faces and is parallel to the reference horizontal plane; and based on the first image, determining rotation offset data of rotation of the tank body around a Y axis, the Y axis being perpendicular to a reference horizontal plane.

Description

Technical Field

[0001] The present disclosure relates to the field of dry quenching coke production systems and installation control technologies thereof, and in particular to an operation status determination method, device, electronic equipment, and medium. Background Art

[0002] The red coke transportation and loading system is a crucial component of the CDQ production system. Its operation involves loading the red coke into a tank, which is then pulled to the bottom of an elevator by a locomotive. The elevator then controls the hook to engage the lugs on the tank, lifting the tank to the top of the CDQ furnace, where the coke is loaded into the furnace via a loading device. During this operation, abnormalities may occur, such as the tank not rotating properly, the locomotive not pulling the tank into position, or the hook not properly engaging the lugs. These conditions can lead to operational interruptions, necessitating a method that automatically detects the tank's position and hook status. Summary of the Invention

[0003] In view of this, the present disclosure provides a method and apparatus for determining a job status, as well as an electronic device and a medium, for at least partially solving the above technical problems.

[0004] In a first aspect, the present disclosure provides a method for determining an operating status, which is applied to a dry quenching production system. The method includes: after controlling a tank body to move to a target position, acquiring a first image through an acquisition device, wherein an acquisition end of the acquisition device is facing a first side surface of a lifting lug when the tank body is in place, and the acquisition device includes an image acquisition unit; based on the first image, determining offset data of the tank body in an X-axis direction, wherein the X-axis is perpendicular to the direction of the acquisition end of the acquisition device and parallel to a reference horizontal plane; based on the first image, determining rotational offset data of the tank body around a Y-axis, wherein the Y-axis is perpendicular to the reference horizontal plane.

[0005] In a second aspect, the present disclosure provides an operating status determination device, which is applied to a dry quenching production system, the device comprising: an acquisition module, configured to acquire a first image through an acquisition device after controlling a tank body to move to a target position, wherein an acquisition end of the acquisition device is facing a first side surface of a lifting lug when the tank body is in place, and the acquisition device comprises an image acquisition unit; a first determination module, configured to determine offset data of the tank body in an X-axis direction based on the first image, wherein the X-axis is perpendicular to the acquisition end of the acquisition device and parallel to a reference horizontal plane; and a first determination module, configured to determine rotational offset data of the tank body around a Y-axis based on the first image, wherein the Y-axis is perpendicular to the reference horizontal plane.

[0006] In a third aspect, the present disclosure provides an electronic device, comprising: a processor, a communication interface, a memory, and a bus, wherein the processor, the communication interface, and the memory communicate with each other via the bus;

[0007] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform operations corresponding to the method described in any one of the first aspects.

[0008] In a fourth aspect, the present disclosure provides a determination machine readable storage medium, on which determination machine instructions are stored. When the determination machine instructions are executed by a processor, the processor executes any one of the methods according to the first aspect.

[0009] In an embodiment of the present application, a first image is captured by a capture device, and based on this first image, the offset data of the tank body in the X-axis direction and the rotational offset data of the tank body about the Y-axis are determined. This effectively detects whether the tank body has offset in the X-axis direction and whether it has rotational offset about the Y-axis, thereby identifying whether the tank body has been precisely moved into position. Furthermore, the image captured by the capture device is combined with point cloud data to identify whether the tank body has been precisely moved into position. Specifically, when capturing the first image, first point cloud data is captured by a laser radar, and based on the first image and the first point cloud data, the offset data of the tank body in the X-axis direction and the rotational offset data about the Y-axis are determined. This improves the accuracy of the result of identifying whether the tank body has been precisely moved into position. Furthermore, the hooking and engagement status of the hook are determined based on the image captured by the capture device, allowing for quick and effective identification of whether the hook is successfully hooked or engaged. Furthermore, in addition to determining the engagement status of the hook based on the image captured by the capture device, the engagement status of the hook can also be determined based on point cloud data captured by the laser radar, providing a redundant solution and enhancing system security. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 4 is a flowchart of a method for determining an operating state of a dry coke quenching production system according to an embodiment of the present disclosure.

[0011] Figure 2 A top view showing a state where the tank body has been moved to a target position is shown.

[0012] Figure 3 A schematic diagram of a first side view of the lifting eye with the tank body in place is shown.

[0013] Figure 4 A side view of the hook is shown in a successfully hooked state.

[0014] Figure 54 is a structural diagram of an operation status determination device of a dry coke quenching production system according to an embodiment of the present disclosure.

[0015] Figure 6 is a structural diagram of an electronic device according to an embodiment of the present disclosure.

[0016] List of reference numerals:

[0017] 10. Collection device; 20. Tank;

[0018] 30. Lifting eye; 31. Lifting eye arm;

[0019] 32. Lifting trunnion; 40. Hook;

[0020] 41. Hook body; 42. Clamping plate;

[0021] 500, operation status determination device; 510, acquisition module;

[0022] 520, first determination module; 530, second determination module;

[0023] 600. Electronic device; 602. Processor;

[0024] 604, communication interface; 606, memory;

[0025] 608. Bus; 610. Program. DETAILED DESCRIPTION

[0026] To make the objectives, technical solutions, and advantages of this application more clearly understood, the present application is further described in detail below with reference to the accompanying drawings and examples. It is apparent that the described embodiments are only a portion of the embodiments of this application, rather than all of them. All other technical solutions derived by those of ordinary skill in the art based on the embodiments in this application fall within the scope of protection of this application.

[0027] Figure 1 The flowchart of the method for determining the working state of an embodiment of the present disclosure is shown, which is mainly applied to the dry quenching production system. Figure 1 As shown, the method includes:

[0028] In S110, after the tank body 20 is controlled to move to the target position, the first image is captured by the acquisition device 10, wherein the acquisition end of the acquisition device 10 is facing the first side surface of the lifting ear 30 when the tank body 20 is in place (such as Figure 3As shown); in S120, based on the first image, the offset data of the tank body 20 in the X-axis direction is determined, wherein the X-axis is perpendicular to the direction of the acquisition end of the acquisition device 10 and parallel to the reference horizontal plane; in S130, based on the first image, the rotation offset data of the tank body 20 around the Y-axis is determined, wherein the Y-axis is perpendicular to the reference horizontal plane. It can be understood that S120 and S130 can be as follows Figure 1 The execution is shown to be performed sequentially, but in other embodiments, it can also be performed in parallel or in a sequential manner.

[0029] Optionally, S120 may further include: determining the pixel center position of the lug axis 32 in the first image; and determining the offset data of the can body 20 in the X-axis direction based on the pixel center position of the lug axis 32, a reference pixel center position of the lug axis 32, and the ratio of the actual size to the pixel size, wherein the reference pixel center position of the lug axis 32 and the ratio of the actual size to the pixel size are determined based on a reference image captured by the acquisition device 10 with the can body 20 in place. The process may further include: calculating the offset between the pixel center position of the lug axis 32 and the reference pixel center position of the lug axis 32, and multiplying the calculated offset by the ratio of the actual size to the pixel size as the offset data of the can body 20 in the X-axis direction. The "pixel center position of the lug axis 32" may be obtained by detecting the first image using a first object detection model to obtain a target frame of the lug axis 32 in the first image, and determining the pixel center position of the lug axis 32 based on the target frame of the lug axis 32.

[0030] Optionally, the aforementioned S130 may further include: determining the pixel size of the two lifting ear arms 31 in the first image; comparing the pixel size of each lifting ear arm 31 with the reference pixel size of the corresponding lifting ear arm 31, and determining the rotation offset data of the tank body 20 around the Y-axis based on the comparison result and the ratio of the actual size to the pixel size, wherein the reference pixel size of the two lifting ear arms 31 is determined based on the aforementioned reference image. The "pixel size of each lifting ear arm 31" can be obtained by the following steps: detecting the first image using a first target detection model to obtain a target frame of the two lifting ear arms 31 in the first image, and determining the pixel size of each lifting ear arm 31 based on the target frame of each lifting ear arm 31.

[0031] Optionally, the aforementioned S110 further includes: collecting first point cloud data using the acquisition device 10, wherein the acquisition device 10 also includes a laser radar. Furthermore, the aforementioned S120 may further include: determining offset data of the tank body 20 in the X-axis direction based on the first image and the first point cloud data. Furthermore, the aforementioned S130 may further include: determining rotation offset data of the tank body 20 around the Y-axis based on the first image and the first point cloud data.

[0032] Furthermore, the aforementioned "determining the offset data of the tank body 20 in the X-axis direction based on the first image and the first point cloud data" can be exemplarily implemented as follows: using the second target detection model to detect the first image to obtain the target frame of the lifting ear 30 in the first image; mapping the pixel position of the target frame of the lifting ear 30 from the image acquisition unit coordinate system to the lidar coordinate system to obtain the point cloud data of the lifting ear 30; segmenting the point cloud data of the lifting ear axis 32 from the point cloud data of the lifting ear 30; calculating the actual center position of the lifting ear axis 32 based on the point cloud data of the lifting ear axis 32; determining the offset data of the tank body 20 in the X-axis direction according to the actual center position of the lifting ear axis 32 and the reference actual center position of the lifting ear axis 32, wherein the reference actual center position of the lifting ear axis 32 is determined based on the reference point cloud data collected by the lidar when the tank body 20 is in place.

[0033] Furthermore, based on the exemplary implementation of "determining the offset data of the tank body 20 in the X-axis direction based on the first image and the first point cloud data" described in the previous paragraph, before "mapping the pixel position of the target frame of the lifting lug 30 from the image acquisition unit coordinate system to the laser radar coordinate system to obtain the point cloud data of the lifting lug 30", it may also exemplarily include: before controlling the tank body 20 to move within the field of view of the laser radar, collecting background point cloud data through the laser radar; using a background subtraction algorithm to subtract the background point cloud data from the first point cloud data to obtain second point cloud data. Furthermore, "mapping the pixel position of the target frame of the lifting lug 30 from the image acquisition unit coordinate system to the laser radar coordinate system to obtain the point cloud data of the lifting lug 30" may exemplarily include: mapping the pixel position of the target frame of the lifting lug 30 from the image acquisition unit coordinate system to the laser radar coordinate system corresponding to the second point cloud data to obtain the point cloud data of the lifting lug 30.

[0034] Preferably, the LiDAR coordinate system is an XYZ coordinate system, with the horizontal axis of the image acquisition unit coordinate system parallel to the X axis and the vertical axis antiparallel to the Y axis. In actual calculations, if the coordinate data involved in the calculation is in different coordinate systems, it is necessary to convert the coordinate data in the different coordinate systems to the same coordinate system before performing the calculation.

[0035] Furthermore, the aforementioned "determining the rotational offset data of the tank body 20 about the Y-axis based on the first image and the first point cloud data" can be implemented in various ways. For example, it can be implemented by: obtaining the remaining point cloud data after segmenting the point cloud data of the lug axis 32 from the point cloud data of the lug 30; performing plane fitting on the remaining point cloud data to obtain a first plane; and calculating the angle between the first plane and the Z-axis to obtain the rotational offset data of the tank body 20 about the Y-axis. For another example, it can also be implemented as follows: obtaining the remaining point cloud data after segmenting the point cloud data of the lifting ear axis 32 from the point cloud data of the lifting ear 30; segmenting the remaining point cloud data to obtain the point cloud range of the two lifting ear arms 31; using the point cloud range of each lifting ear arm 31, calculating the actual average Z coordinate of each lifting ear arm 31, wherein the Z axis is parallel to the direction facing the acquisition end of the acquisition device 10; comparing the actual average Z coordinate of each lifting ear arm 31 with the reference average Z coordinate, and determining the rotational offset data of the tank body 20 around the Y axis based on the comparison result of the average Z coordinates of the two lifting ear arms 31, wherein the reference average Z coordinate is determined based on the reference point cloud data collected by the lidar when the tank body 20 is in place.

[0036] The specific implementation of "segmenting the remaining point cloud data to obtain the point cloud ranges of the two lifting ear arms 31" described in the previous paragraph can be varied. For example, based on the pixel center positions of the lifting ear axis 32 obtained previously, algorithms such as straight-through filtering can be used to segment the point cloud ranges of the two lifting ear arms 31. Another example is to use the previously obtained target frame of the lifting ear arm and map the vertex coordinates of the target frame of the lifting ear arm from the image acquisition unit coordinate system to the laser radar coordinate system to segment the point cloud ranges of the two lifting ear arms 31.

[0037] It can be understood that in this embodiment, when the tank body 20 is in place, the offset of the tank body 20 in the X-axis direction is within an allowable range, and the rotation offset around the Y-axis is within an allowable range. In addition, the tank body moves on a reference horizontal plane, which is usually the ground. In addition, in each embodiment of the present disclosure, in an implementation method that does not require the collection of point cloud data, the acquisition device 10 can be a separate image acquisition unit; in an implementation method that requires the collection of point cloud data, the acquisition device 10 can be composed of both an image acquisition unit and a laser radar, such as Figure 2 shown.

[0038] Optionally, the operation status determination method may further include the step of "determining the hook 40 status of the hook 40". The "determining the hook 40 status of the hook 40" may be implemented, for example, as follows: in the process of controlling the hook 40 to perform the hook 40 action, acquiring at least one frame of the second image; detecting the second image using a third target detection model to obtain a target frame of the hook body 41 and a target frame of the engaging plate 42 of the hook 40; determining the pixel center position of the hook body 41 and the pixel center position of the engaging plate 42 based on the target frame of the hook body 41 and the target frame of the engaging plate 42 respectively; calculating the offset in the X-axis direction between the pixel center position of the hook body 41 and the pixel center position of the hanging ear axis region and the offset in the X-axis direction between the pixel center position of the engaging plate 42 and the pixel center position of the hanging ear axis, and determining the hook 40 status of the hook 40 based on the calculated offset and the aforementioned ratio of the actual size to the pixel size.

[0039] It is understood that the offset data involved in this embodiment includes, for example, an offset state and / or an offset amount. In practical applications, for example, the determined offset data can be output to a display device; or the offset amount can be compared with a preset value, and a selective alarm can be issued based on the comparison result.

[0040] Optionally, the method for determining the operation status may further include the step of "determining the hooked state of the hook 40." The specific implementation methods for "determining the hooked state of the hook 40" are various. For example, the "determining the hooked state of the hook 40" may be specifically implemented as follows: after controlling the hook 40 to perform the hooking action, acquiring at least one third image; detecting the third image using a third target detection model to obtain a target frame of the hook body 41 and a target frame of the engaging plate 42; determining the pixel center position of the hook body 41 and the pixel center position of the engaging plate 42 based on the target frame of the hook body 41 and the target frame of the engaging plate 42, respectively; calculating the offset between the pixel center position of the hook body 41 and the pixel center position of the engaging plate 42, and determining the hooked state of the hook 40 based on the calculated offset and the aforementioned ratio of the actual size to the pixel size. Further, if the calculated offset is greater than a preset value, the hooked state of the hook 40 is determined to be "failed," otherwise it is determined to be "successful." Figure 4 An exemplary view of the hook 40 in a successfully hooked state along the X-axis direction is shown.

[0041] For another example, "determining the hooked state of the hook 40" can also be specifically implemented as follows: after controlling the tank body 20 to move to the target position, first point cloud data is collected by using a laser radar; when controlling the hook 40 to perform the hooking process, third point cloud data is collected by using the laser radar, the first point cloud data in the third point cloud data is removed to obtain fourth point cloud data, and the fourth point cloud data is segmented, for example, by using a clustering algorithm, two-step RANSAC plane fitting, or other segmentation methods to obtain point cloud data of the hook body 41 and the engaging plate 42 of the hook 40 respectively; based on the point cloud data of the hook body 41 and the engaging plate 42 of the hook 40, the X coordinate and Z coordinate of the actual center position of the hook body 41 of the hook 40 and the X coordinate and Z coordinate of the actual center position of the engaging plate 42 are calculated respectively; the X coordinate and Z coordinate of the actual center position of the hook body 41 of the hook 40 and the X coordinate and Z coordinate of the actual center position of the engaging plate 42 are compared respectively with the X coordinate and Z coordinate of the actual center position of the lifting ear shaft 32; and the hooked state of the hook 40 is determined according to the comparison result. Furthermore, if any comparison result is greater than a preset value, the hooking state of the hook 40 is determined to be "failed," otherwise it is determined to be "successful." It will be understood that the "fourth point cloud data" described in this paragraph includes the point cloud data of the hook body 41 and the engaging plate 42, and the two need to be separated. During the hooking process, the point cloud data of the hook body 41 and the point cloud data of the engaging plate 42 are separated and not in the same plane. After the hooking is completed, the point cloud data of the hook body 41 and the point cloud data of the engaging plate 42 are in the same plane, but there is a gap between them.

[0042] Furthermore, the "determining the hooking state of the hook 40" can also be specifically implemented as follows: after controlling the tank body 20 to move to the target position, the first point cloud data is acquired by the laser radar; in the process of controlling the hook 40 to perform the hooking, the third point cloud data is acquired by the laser radar, the first point cloud data in the third point cloud data is removed to obtain the fifth point cloud data, and the fifth point cloud data is segmented, for example, by using a clustering algorithm, two-times RANSAC plane fitting and other segmentation methods to obtain the point cloud data of the hook body 41 of the hook 40 and the point cloud data of the engaging plate 42, respectively, based on the point cloud data. The X coordinates of the actual center positions of the hook body 41 and the engaging plate 42 of the hook 40 are calculated based on the point cloud data of the hook body 41 and the point cloud data of the engaging plate 42. The actual size of the gap between the hook body 41 and the engaging plate 42 in the fifth point cloud data is calculated. The X coordinates of the actual center positions of the hook body 41 and the engaging plate 42 of the hook 40 are compared with the X coordinates of the actual center positions of the trunnion shaft 32. The engaged state of the hook 40 is determined based on the comparison results and the calculated actual size of the gap. Further, if the comparison result of the X coordinates is greater than a preset value or the calculated actual size of the gap is greater than another preset value, the engaged state of the hook 40 is determined to be "failed." If the comparison result of the X coordinates is not greater than the preset value or the calculated actual size of the gap is not greater than another preset value, the engaged state of the hook 40 is determined to be "successful."

[0043] In addition, the first target detection model, the second target detection model, and the third target detection model involved in this embodiment can, for example, utilize an acquisition device to collect images in different environments and different working conditions, and utilize a labeling tool to label the detection targets in the image, and use the labeled images to train the YOLO series neural network to obtain them. Taking the first target detection model as an example, the first target detection model can utilize a labeling tool to label the lifting ear shaft and the two lifting ear arms in the image, and use the labeled images to train the YOLO series neural network to obtain them. The second target detection model and the third target detection model are similar. In addition, in this embodiment, if the target detection model fails to detect the target object, the target object can be detected by methods such as 3D target detection or target matching. And, it can be understood that, Figure 2 、 Figure 3 and Figure 4 The shapes of the tank body 20, the lifting lug 30, and the hook 40 shown are merely exemplary, and other shapes without changing their components should also fall within the scope of protection of the present disclosure.

[0044] In order to implement the method for determining the operating state of the dry coke quenching production system of the above embodiment, the embodiment of the present disclosure further provides an operating state determining device for the dry coke quenching production system, such as Figure 5As shown, the device 500 includes a collection module 510, a first determination module 520, and a second determination module 530. It should be noted that since the following embodiments are intended to implement the aforementioned method embodiments, each module in the job status determination system is designed to implement each step of the aforementioned method. Therefore, the present invention is not limited to the following embodiments, and any device or module that can implement the aforementioned method should be included in the scope of protection of the present invention.

[0045] In this embodiment, the acquisition module 510 is used to acquire a first image through the acquisition device 10 after controlling the tank body 20 to move to the target position, wherein the acquisition end of the acquisition device 10 is facing the first side surface of the lifting ear when the tank body 20 is in place (such as Figure 3 As shown), the acquisition device 10 includes an image acquisition unit.

[0046] In this embodiment, the first determination module 520 is configured to determine the offset data of the tank body 20 in the X-axis direction based on the first image, where the X-axis is perpendicular to the acquisition end of the acquisition device 10 and parallel to the reference horizontal plane. Optionally, the first determination module 520 is further configured to: determine the pixel center position of the lug axis 32 in the first image; and determine the offset data of the tank body 20 in the X-axis direction based on the pixel center position of the lug axis 32, a reference pixel center position of the lug axis 32, and a ratio of the actual size to the pixel size, where the reference pixel center position of the lug axis 32 and the ratio of the actual size to the pixel size are determined based on a reference image captured by the acquisition device 10 with the tank body 20 in place. The first determination module 520 illustratively detects the first image using a first object detection model to obtain a target frame of the lug axis 32 in the first image, and determines the pixel center position of the lug axis 32 based on the target frame of the lug axis 32.

[0047] In this embodiment, the second determination module 530 is used to determine the rotation offset data of the tank body 20 rotating around the Y axis based on the first image, wherein the Y axis is perpendicular to the reference horizontal plane. Optionally, the second determination module 530 is further used to: determine the pixel size of the two lifting ear arms 31 in the first image; respectively compare the pixel size of each lifting ear arm 31 with the reference pixel size of the corresponding lifting ear arm 31, and determine the rotation offset data of the tank body 20 rotating around the Y axis based on the comparison result and the ratio of the actual size to the pixel size, wherein the reference pixel size of the two lifting ear arms 31 is determined based on the reference image. Among them, the second determination module 530 exemplarily uses the first target detection model to detect the first image to obtain the target frames of the two lifting ear arms 31, and determines the pixel size of each lifting ear arm 31 based on the target frame of each lifting ear arm 31.

[0048] Optionally, the acquisition module 510 is further configured to acquire first point cloud data using the acquisition device 10 when acquiring the first image, where the acquisition device 10 includes a laser radar. Furthermore, the first determination module 520 is further illustratively configured to determine offset data of the tank body 20 in the X-axis direction based on the first image and the first point cloud data. The first determination module 520 further uses the second target detection model to detect the first image to obtain the target frame of the lifting ear 30 in the first image, maps the pixel position of the target frame of the lifting ear 30 from the image acquisition unit coordinate system to the lidar coordinate system to obtain the point cloud data of the lifting ear 30, segments the point cloud data of the lifting ear axis 32 from the point cloud data of the lifting ear 30, calculates the actual center position of the lifting ear axis 32 based on the point cloud data of the lifting ear axis 32, and determines the offset data of the tank body 20 in the X-axis direction according to the actual center position of the lifting ear axis 32 and the reference actual center position of the lifting ear axis 32, wherein the reference actual center position of the lifting ear axis 32 is determined based on the reference point cloud data collected by the lidar when the tank body 20 is in place.

[0049] Based on the exemplary implementation described in the previous paragraph, the first determination module 520 is further configured to, before mapping the pixel position of the target frame of the lifting lug 30 from the image acquisition unit coordinate system to the lidar coordinate system to obtain point cloud data of the lifting lug 30 and before controlling the tank body 20 to move within the field of view of the lidar, collect background point cloud data using the lidar and, using a background subtraction algorithm, subtract the background point cloud data from the first point cloud data to obtain second point cloud data. Furthermore, the first determination module 520 is further configured to map the pixel position of the target frame of the lifting lug 30 from the image acquisition unit coordinate system to the lidar coordinate system corresponding to the second point cloud data to obtain point cloud data of the lifting lug 30.

[0050] Furthermore, the second determination module 530 can also be exemplarily used to determine the rotation offset data of the tank body 20 rotating about the Y-axis based on the first image and the first point cloud data. Specifically, on one hand, the second determination module 530 can also be exemplarily used to: obtain the remaining point cloud data after segmenting the point cloud data of the lifting lug 30 from the point cloud data of the lifting lug axis 32; perform plane fitting on the remaining point cloud data to obtain a first plane; and calculate the angle between the first plane and the Z-axis to obtain the rotation offset data of the tank body 20 rotating about the Y-axis, wherein the Z-axis is parallel to the direction facing the acquisition end of the acquisition device 10, and the Y-axis is perpendicular to the reference horizontal plane. On the other hand, the second determination module 530 can also be used to: obtain the remaining point cloud data after the point cloud data of the lifting ear axis 32 is segmented from the point cloud data of the lifting ear 30; segment the remaining point cloud data to obtain the point cloud range of the two lifting ear arms 31; use the point cloud range of each lifting ear arm 31 to calculate the actual average Z coordinate of each lifting ear arm 31, wherein the Z axis is parallel to the direction facing the acquisition end of the acquisition device 10; compare the actual average Z coordinate of each lifting ear arm 31 with the reference average Z coordinate, and determine the rotational offset data of the tank body 20 around the Y axis based on the comparison result of the average Z coordinates of the two lifting ear arms 31, wherein the reference average Z coordinate is determined based on the reference point cloud data collected by the lidar when the tank body 20 is in place.

[0051] Optionally, the apparatus 500 may further include a third determination module. In an exemplary implementation, the third determination module is configured to: acquire at least one frame of a second image while controlling the hook 40 to perform the hook 40 action; detect the second image using a third target detection model to obtain a target frame of the hook body 41 and a target frame of the engaging plate 42 of the hook 40; determine the pixel center position of the hook body 41 and the pixel center position of the engaging plate 42 based on the target frame of the hook body 41 and the target frame of the engaging plate 42, respectively; calculate an offset in the X-axis direction between the pixel center position of the hook body 41 and the pixel center position of the engaging plate 42 and the pixel center position of the ear shaft, and determine the hook 40 state of the hook 40 based on the calculated offset and the ratio of the actual size to the pixel size.

[0052] Optionally, the apparatus 500 may further include a fourth determining module.

[0053] In one exemplary implementation, the acquisition module 510 is further configured to acquire at least one third image after controlling the hook 40 to perform the hooking action. Furthermore, the fourth determination module is configured to detect the third image using a third target detection model, obtain a target frame of the hook body 41 and a target frame of the engaging plate 42, determine the pixel center positions of the hook body 41 and the pixel center positions of the engaging plate 42 based on the target frames of the hook body 41 and the engaging plate 42, respectively, calculate an offset between the pixel center positions of the hook body 41 and the pixel center positions of the engaging plate 42, and determine the hooked state of the hook 40 based on the calculated offset and a ratio of the actual size to the pixel size.

[0054] In another exemplary implementation, the acquisition module 510 is also used to acquire first point cloud data through a laser radar after controlling the tank body 20 to move to the target position, and to acquire third point cloud data through a laser radar while controlling the hook 40 to perform the hooking process. In addition, the fourth determination module is used to remove the first point cloud data from the third point cloud data to obtain fourth point cloud data, segment the fourth point cloud data to obtain point cloud data of the hook body 41 and the locking plate 42 of the hook 40 respectively, calculate the X coordinate and Z coordinate of the pixel center position of the hook body 41 of the hook 40 and the X coordinate and Z coordinate of the pixel center position of the locking plate 42 based on the point cloud data of the hook body 41 and the locking plate 42 respectively, compare the X coordinate and Z coordinate of the pixel center position of the hook body 41 of the hook 40 and the X coordinate and Z coordinate of the pixel center position of the locking plate 42 with the X coordinate and Z coordinate of the pixel center position of the ear shaft 32 respectively, and judge the hook state of the hook 40 according to the comparison result, wherein the Y axis is perpendicular to the reference horizontal plane, and the Z axis is parallel to the direction facing the acquisition end of the acquisition device 10.

[0055] In another exemplary implementation, the acquisition module 510 is also used to obtain first point cloud data through laser radar after controlling the tank body 20 to move to the target position, and to obtain third point cloud data through laser radar when controlling the hook 40 to perform the hooking process. In addition, the fourth determination module is used to remove the first point cloud data from the third point cloud data to obtain fifth point cloud data, segment the fifth point cloud data to obtain point cloud data of the hook body 41 of the hook 40 and point cloud data of the engaging plate 42, calculate the X coordinate of the pixel center position of the hook body 41 of the hook 40 and the X coordinate of the pixel center position of the engaging plate 42 based on the point cloud data of the hook body 41 of the hook 40 and the point cloud data of the engaging plate 42, calculate the actual size of the gap between the hook body 41 and the engaging plate 42 in the fifth point cloud data, compare the X coordinate of the pixel center position of the hook body 41 of the hook 40 and the X coordinate of the pixel center position of the engaging plate 42 with the X coordinate of the pixel center position of the lifting ear shaft 32, and determine the hooked state of the hook 40 based on the comparison result and the calculated actual size of the gap.

[0056] It should be noted that the job status determination method of the aforementioned embodiment is a method embodiment corresponding to the job status determination apparatus 500 of this embodiment. The job status determination apparatus 500 of this embodiment can be implemented in conjunction with the job status determination method of the aforementioned embodiment. The relevant technical details mentioned in the job status determination method of the aforementioned embodiment are still valid in the job status determination apparatus 500 of this embodiment and will not be repeated here to avoid repetition.

[0057] Figure 6 This is a schematic diagram of an electronic device according to an embodiment of the present application. The specific embodiment of the present application does not limit the specific implementation of the electronic device. Figure 6 The electronic device 600 provided in the embodiment of the present application includes: a processor 602, a communications interface 604, a memory 606, and a bus 608.

[0058] The processor 602 , the communication interface 604 , and the memory 606 communicate with each other via a bus 608 .

[0059] The communication interface 604 is used to communicate with other electronic devices or servers.

[0060] The processor 602 is configured to execute the program 610 , and specifically may execute the relevant steps in the above method embodiment.

[0061] Specifically, the program 610 may include program codes, which include computer operation instructions.

[0062] Processor 602 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application. The one or more processors included in the smart device may be processors of the same type, such as one or more CPUs, or may be processors of different types, such as one or more CPUs and one or more ASICs.

[0063] The memory 606 is used to store the program 610. The memory 606 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0064] The program 610 may be specifically configured to enable the processor 602 to execute the method in any of the aforementioned embodiments.

[0065] The specific implementation of each step in program 610 can be found in the corresponding descriptions of the corresponding steps and units in the above-mentioned method embodiments, and will not be repeated here. Those skilled in the art will clearly understand that for the convenience and brevity of description, the specific working processes of the above-mentioned devices and modules can refer to the corresponding process descriptions in the above-mentioned method embodiments, and will not be repeated here.

[0066] This application also provides a computer-readable storage medium storing instructions for causing a machine to execute the method described herein. Specifically, a system or device equipped with a storage medium can be provided, wherein the storage medium stores software program code that implements the functions of any of the above-described embodiments, and a computer (or CPU or MPU) of the system or device can read and execute the program code stored in the storage medium.

[0067] In this case, the program code read from the storage medium itself can realize the function of any one of the above embodiments, so the program code and the storage medium storing the program code constitute part of this application.

[0068] Examples of storage media for providing program code include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, the program code can be downloaded from a server computer via a communication network.

[0069] An embodiment of the present application also provides a computer program product, including computer instructions, which instruct a computing device to perform any corresponding operation in the above-mentioned multiple method embodiments.

[0070] It should be pointed out that, according to the needs of implementation, the various components / steps described in the embodiments of the present application can be split into more components / steps, or two or more components / steps or partial operations of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of the present application.

[0071] The above-mentioned method according to the embodiment of the present application can be implemented in hardware, firmware, or can be implemented as software or computer code that can be stored in a recording medium (such as CD ROM, RAM, floppy disk, hard disk or magneto-optical disk), or can be implemented as computer code that is originally stored in a remote recording medium or a non-temporary machine-readable medium downloaded via a network and will be stored in a local recording medium, so that the method described herein can be stored in such software processing on a recording medium using a general-purpose computer, a special-purpose processor or programmable or special-purpose hardware (such as ASIC or FPGA). It can be understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component (e.g., RAM, ROM, flash memory, etc.) that can store or receive software or computer code, and when the software or computer code is accessed and executed by a computer, a processor or hardware, the method described herein is implemented. In addition, when a general-purpose computer accesses the code for implementing the method shown here, the execution of the code converts the general-purpose computer into a special-purpose computer for executing the method shown here.

[0072] It should be noted that not all steps and modules in the above processes and system structure diagrams are required, and certain steps or modules can be omitted according to actual needs. The execution order of each step is not fixed and can be adjusted as needed. The system structure described in the above embodiments can be a physical structure or a logical structure, that is, some modules may be implemented by the same physical entity, or some modules may be implemented by multiple physical entities, or may be implemented by certain components in multiple independent devices.

[0073] Nouns and pronouns referring to people in this patent application are not limited to a specific gender. The terms "comprises," "comprising," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a" does not preclude the presence of additional identical elements in the process, method, article, or apparatus comprising the element.

[0074] In the above embodiments, the hardware module can be implemented mechanically or electrically. For example, a hardware module can include a permanent dedicated circuit or logic (such as a dedicated processor, FPGA or ASIC) to complete the corresponding operation. The hardware module can also include programmable logic or circuits (such as a general-purpose processor or other programmable processors), which can be temporarily set by software to complete the corresponding operation. The specific implementation method (mechanical method, or dedicated permanent circuit, or temporarily set circuit) can be determined based on cost and time considerations.

[0075] The present application has been presented and described in detail above through the accompanying drawings and preferred embodiments. However, the present application is not limited to these disclosed embodiments. Based on the above multiple embodiments, those skilled in the art can know that the code review methods in the above different embodiments can be combined to obtain more embodiments of the present application, and these embodiments are also within the scope of protection of the present application.

Claims

1. A method for determining an operation state, applied to a dry quenching coke production system, characterized in that: The method comprises: After the tank body (20) is controlled to move to a target position, a first image is collected by a collection device (10), wherein a collection end of the collection device (10) faces a first side surface of the lifting lug (30) when the tank body (20) is in position, and the collection device (10) includes an image collection unit; Based on the first image, determining the offset data of the tank body (20) in the X-axis direction, wherein the X-axis is perpendicular to the direction of the acquisition end of the acquisition device (10) and parallel to the reference horizontal plane; Based on the first image, rotation offset data of the tank body (20) rotating around the Y axis is determined, wherein the Y axis is perpendicular to a reference horizontal plane.

2. The method according to claim 1, characterized in that The method of determining the offset data of the tank body (20) in the X-axis direction based on the first image further includes: Determining the pixel center position of the trunnion (32) in the first image; The offset data of the tank body (20) in the X-axis direction is determined based on the pixel center position of the lifting lug (32), the reference pixel center position of the lifting lug (32), and the ratio of the actual size to the pixel size, wherein the reference pixel center position of the lifting lug (32) and the ratio of the actual size to the pixel size are determined based on a reference image of the tank body (20) in an in-place state acquired by the acquisition device (10).

3. The method according to claim 2, characterized in that The method of determining the rotation offset data of the tank body (20) around the Y axis based on the first image further comprises: Determining the pixel size of the two ear arms (31) in the first image; The pixel size of each lug arm (31) is compared with the reference pixel size of the corresponding lug arm (31), and the rotation offset data of the tank body (20) around the Y axis is determined based on the comparison result and the ratio of the actual size to the pixel size, wherein the reference pixel sizes of the two lug arms (31) are determined based on the reference image.

4. The method according to claim 3, characterized in that The pixel center position of the ear axis (32) and the pixel size of each ear arm (31) are obtained by the following steps: Detecting the first image using a first target detection model to obtain a target frame of the trunnion shaft (32) and a target frame of the two trunnion arms (31) in the first image; Determining a pixel center position of the trunnion (32) based on a target frame of the trunnion (32); The pixel size of each ear arm (31) is determined based on the target frame of each ear arm (31).

5. The method according to claim 1, wherein After controlling the tank body (20) to move to the target position, when collecting the first image through the collection device (10), the method further includes: collecting first point cloud data through the collection device (10), wherein the collection device (10) also includes a laser radar; Determining the offset data of the tank body (20) in the X-axis direction based on the first image further includes: determining the offset data of the tank body (20) in the X-axis direction based on the first image and the first point cloud data; The method of determining the rotation offset data of the tank body (20) rotating around the Y axis based on the first image further includes: determining the rotation offset data of the tank body (20) rotating around the Y axis based on the first image and the first point cloud data.

6. The method according to claim 5, characterized in that The method of determining the offset data of the tank body (20) in the X-axis direction based on the first image and the first point cloud data further includes: Detecting the first image using a second target detection model to obtain a target frame of the ear (30) in the first image; Mapping the pixel position of the target frame of the lifting lug (30) from the image acquisition unit coordinate system to the laser radar coordinate system to obtain point cloud data of the lifting lug (30); Segmenting point cloud data of the lifting lug shaft (32) from the point cloud data of the lifting lug (30); Calculating the actual center position of the trunnion (32) based on the point cloud data of the trunnion (32); The offset data of the tank body (20) in the X-axis direction is determined based on the actual center position of the lifting lug shaft (32) and the reference actual center position of the lifting lug shaft (32), wherein the reference actual center position of the lifting lug shaft (32) is determined based on the reference point cloud data collected by the collection device (10) when the tank body (20) is in place.

7. The method according to claim 6, characterized in that Before mapping the pixel position of the target frame of the lifting ear (30) from the image acquisition unit coordinate system to the laser radar coordinate system to obtain the point cloud data of the lifting ear (30), the method further includes: Before controlling the tank (20) to move within the field of view of the acquisition device (10), background point cloud data is acquired by the acquisition device (10); The background point cloud data is subtracted from the first point cloud data using a background subtraction algorithm to obtain second point cloud data.

8. The method according to claim 7, characterized in that The step of mapping the pixel position of the target frame of the lifting ear (30) from the image acquisition unit coordinate system to the laser radar coordinate system to obtain point cloud data of the lifting ear (30) further comprises: The pixel position of the target frame of the lifting lug (30) is mapped from the image acquisition unit coordinate system to the laser radar coordinate system corresponding to the second point cloud data to obtain the point cloud data of the lifting lug (30).

9. The method according to claim 6, characterized in that The method of determining the rotation offset data of the tank body (20) around the Y axis based on the first image and the first point cloud data further comprises: Obtaining remaining point cloud data after segmenting the point cloud data of the lifting lug (30) into the point cloud data of the lifting lug shaft (32); Performing plane fitting on the remaining point cloud data to obtain a first plane; The angle between the first plane and the Z axis is calculated to obtain the rotation offset data of the tank body (20) rotating around the Y axis, wherein the Z axis is parallel to the direction facing the collection end of the collection device (10), and the Y axis is perpendicular to the reference horizontal plane.

10. The method according to claim 6, characterized in that The method of determining the rotation offset data of the tank body (20) around the Y axis based on the first image and the first point cloud data further comprises: Obtaining remaining point cloud data after segmenting the point cloud data of the lifting lug (30) into the point cloud data of the lifting lug shaft (32); Segmenting the remaining point cloud data to obtain point cloud ranges of the two lifting lug arms (31); Calculating the actual average Z coordinate of each lifting lug arm (31) using the point cloud range of each lifting lug arm (31), wherein the Z axis is parallel to the direction in which the collection end of the collection device (10) faces; The actual average Z coordinate of each lifting lug arm (31) is compared with a reference average Z coordinate, and the rotation offset data of the tank body (20) rotating around the Y axis is determined based on the comparison result of the average Z coordinates of the two lifting lug arms (31), wherein the reference average Z coordinate is determined based on the reference point cloud data collected by the collection device (10) when the tank body (20) is in place, and the Y axis is perpendicular to the reference horizontal plane.

11. The method according to any one of claims 1 to 10, characterized in that The method further comprises: During the process of controlling the hook (40) to perform the hooking action, at least one frame of the second image is acquired; Detecting the second image using a third target detection model to obtain a target frame of the hook body (41) and a target frame of the engaging plate (42) of the hook (40); Determining the pixel center position of the hook body (41) and the pixel center position of the engaging plate (42) based on the target frame of the hook body (41) and the target frame of the engaging plate (42) respectively; The offsets of the pixel center positions of the hook body (41) and the pixel center positions of the hanging ear shaft (32) in the X-axis direction and the offsets of the pixel center positions of the locking plate (42) and the pixel center positions of the hanging ear shaft (32) in the X-axis direction are calculated respectively, and the hook state of the hook (40) is determined according to the calculated offsets and the ratio of the actual size to the pixel size, wherein the ratio of the actual size to the pixel size is determined based on a reference image of the tank body (20) in the in-place state acquired by the acquisition device (10).

12. The method according to claim 11, characterized in that The method further comprises: After controlling the hook (40) to perform a hooking action, acquiring at least one third image; Detecting the third image using a third target detection model to obtain a target frame of the hook body (41) and a target frame of the engaging plate (42) of the hook (40); Determining the pixel center position of the hook body (41) and the pixel center position of the engaging plate (42) based on the target frame of the hook body (41) and the target frame of the engaging plate (42) respectively; The offset between the pixel center position of the hook body (41) and the pixel center position of the engaging plate (42) is calculated, and the hooking state of the hook (40) is determined based on the calculated offset and the ratio of the actual size to the pixel size, wherein the ratio of the actual size to the pixel size is determined based on a reference image of the tank body (20) in the in-position state acquired by the acquisition device (10).

13. The method according to any one of claims 1 to 10, characterized in that The method further comprises: After controlling the tank body (20) to move to the target position, first point cloud data is collected by the collection device (10), wherein the collection device (10) includes a laser radar; During the process of controlling the hook (40) to perform the hooking, Acquiring third point cloud data through the acquisition device (10); removing the first point cloud data from the third point cloud data to obtain fourth point cloud data; Segmenting the fourth point cloud data to obtain point cloud data of the hook body (41) and the clamping plate (42) of the hook (40); Based on the point cloud data of the hook body (41) and the engaging plate (42) of the hook (40), the X coordinate and the Z coordinate of the actual center position of the hook body (41) and the engaging plate (42) of the hook (40) are calculated, wherein the Y axis is perpendicular to the reference horizontal plane, and the Z axis is parallel to the direction in which the collecting end of the collecting device (10) faces; The X coordinates and Z coordinates of the actual center positions of the hook body (41) and the locking plate (42) of the hook (40) are respectively compared with the X coordinates and Z coordinates of the actual center position of the hanging ear shaft (32), and the hooking state of the hook (40) is determined based on the comparison results.

14. The method according to any one of claims 1 to 10, characterized in that The method further comprises: After controlling the tank body (20) to move to the target position, first point cloud data is acquired by the acquisition device (10), wherein the acquisition device (10) includes a laser radar; During the process of controlling the hook (40) to perform the hooking, Acquiring third point cloud data through the acquisition device (10); removing the first point cloud data from the third point cloud data to obtain fifth point cloud data; Segmenting the fifth point cloud data to obtain point cloud data of the hook body (41) and the clamping plate (42) of the hook (40); Calculating the X coordinates of the actual center positions of the hook body (41) and the engaging plate (42) of the hook (40) based on the point cloud data of the hook body (41) and the engaging plate (42), and calculating the actual size of the gap between the hook body (41) and the engaging plate (42) in the fifth point cloud data; The X coordinates of the actual center positions of the hook body (41) and the locking plate (42) of the hook (40) are compared with the X coordinates of the actual center position of the ear shaft (32), and the hooked state of the hook (40) is determined based on the comparison result and the calculated actual size of the gap.

15. An operation status determination device, applied to a dry quenching coke production system, characterized in that: The device (500) comprises: A collection module (510) is used to collect a first image through a collection device (10) after controlling the tank body (20) to move to a target position, wherein a collection end of the collection device (10) faces a first side surface of the lifting lug (30) when the tank body (20) is in place, and the collection device (10) includes an image collection unit; a first determining module (520) for determining offset data of the tank body (20) in an X-axis direction based on the first image, wherein the X-axis is perpendicular to the direction of the acquisition end of the acquisition device (10) and parallel to a reference horizontal plane; The second determination module (530) is used to determine the rotation offset data of the tank body (20) rotating around the Y axis based on the first image, wherein the Y axis is perpendicular to the reference horizontal plane.

16. An electronic device (600), comprising: A processor (602), a communication interface (604), a memory (606), and a bus (608), wherein the processor (602), the communication interface (604), and the memory (606) communicate with each other via the bus (608); The memory (606) is used to store at least one executable instruction, and the executable instruction enables the processor (602) to perform an operation corresponding to the method according to any one of claims 1 to 14. 17 . A determination machine readable storage medium, wherein determination machine instructions are stored on the determination machine readable storage medium, and when the determination machine instructions are executed by a processor, the processor is caused to execute the method according to any one of claims 1 to 14.

Images