A method and device for configuring a work order for roller inspection, a storage medium and an apparatus

Abstract

The application provides a roller inspection work order configuration method and device, a storage medium and equipment. Center point coordinates and scale information of a target roller detection box corresponding to a target visual angle in a current frame image collected by an inspection robot are acquired. If the current frame image does not meet qualified conditions, the inspection robot is adjusted. After the qualified conditions are met, the current horizontal angle and current pitch angle of a holder, the current height of a lifting arm, and the current focal length of an inspection camera are recorded to a to-be-configured work order, and an inspection work order corresponding to an inspection point is obtained. The qualified conditions are used as constraints to adjust the horizontal angle and pitch angle of the holder, the height of the lifting arm, and the focal length of the inspection camera. Not only are all mechanical errors and track deformation errors absorbed and digested, but also smooth and stable holder compensation is achieved under the condition of variable focal length, so that the picture taken by the inspection robot for each roller has a highly consistent composition standard, and the subsequent data reliability and configuration efficiency are improved.

Description

Technical Field

[0001] This invention relates to the field of monitoring, and more specifically, to a method, apparatus, storage medium, and equipment for configuring idler roller inspection work orders. Background Technology

[0002] Belt conveyors are indispensable continuous conveying equipment for bulk materials in heavy industries such as mining, ports, steel metallurgy, and thermal power generation. In the entire conveyor line, idlers, as the core component carrying the belt and materials, directly determine the safety and efficiency of the production line due to their operating status (such as whether they operate smoothly, whether abnormal wear occurs, whether bearings overheat or foreign objects are stuck). To achieve real-time and automated monitoring of the operating status of the numerous idlers, as well as the upper and return belts, track-mounted intelligent inspection robot systems have been widely introduced in industrial settings.

[0003] Conventional intelligent inspection robots for rail conveyors typically employ a suspended design. Due to the robot's considerable vertical height, they are generally mounted above belt conveyors for optimal monitoring coverage. However, modern belt conveyors, for dust and rain protection and environmental compliance, usually have enclosed or semi-enclosed rain covers installed above them. While these physical protective covers protect the materials, they also severely obstruct the intelligent inspection robot's view, limiting its monitoring field of vision and preventing it from effectively penetrating the rain cover to monitor the actual operating status of the internal rollers.

[0004] To overcome this field-of-view obstacle, a new intelligent inspection robot architecture for belt conveyors has emerged in recent years. This design deploys the inspection robot and its I-shaped hoisting track within the confined space between the upper and return belts of a belt conveyor, placing it at the same level as the side intermediate frames. This design allows the robot to monitor the condition of idlers from inside the conveyor. Such robots are typically equipped with a walking mechanism (such as a drive motor, drive wheels, and tensioning pulleys), a control unit, an ultrasonic anti-collision module, a lighting module, and a multi-DOF gimbal camera with visible light and infrared thermal imaging sensors. Some high-end models also feature a second inspection object or a lifting robotic arm to expand the inspection field of view, enabling the coverage of blind spots by adjusting height and angle.

[0005] Under this architecture, the robot automatically inspects along the track, stops when it reaches the designated physical mileage node, and calls the horizontal yaw angle, pitch angle and zoom parameters of the gimbal camera according to the pre-entered work order to perform image acquisition and infrared temperature measurement on the specific idler roller.

[0006] While the belt-mounted robot solves the problem of rain cover obstructing the macroscopic view by lowering its physical position, existing technologies still reveal extremely high initial calibration costs for a massive number of work unit poses (inspection points) when actually implementing refined inspections of idlers and managing work order locations. Specifically, a medium to large belt conveyor often stretches for several kilometers and contains thousands of idler groups. During the system initialization and deployment phase, an independent inspection work order must be created for each idler requiring monitoring. Existing technology relies on maintenance personnel in the control room facing the back-end terminal, manually controlling the robot's movement position using joysticks or virtual buttons, blindly adjusting the pan-tilt head's horizontal and vertical angles, and the camera's focal length, in order to completely capture the target idler in the image. This purely manual open-loop trial-and-error process results in extremely long creation times for hundreds or thousands of pose points, not only severely delaying the on-site deployment progress but also causing significant differences in the standardization of the basic data for subsequent algorithm recognition due to inconsistent subjective standards among different personnel. Summary of the Invention

[0007] The purpose of this invention is to provide a method, apparatus, storage medium, and equipment for configuring idler roller inspection work orders, so as to improve the above-mentioned problems.

[0008] To achieve the above objectives, the technical solutions adopted in the embodiments of the present invention are as follows: In a first aspect, embodiments of the present invention provide a method for configuring inspection work orders for idler rollers, which is applied to an inspection robot. The inspection robot includes an inspection camera deployed on a gimbal, the gimbal being mounted on a lifting arm. When the inspection robot travels to a preset inspection point, the method includes: The coordinates and scale information of the center point of the target roller detection frame corresponding to the target viewpoint in the current frame image collected by the inspection robot are obtained, wherein the scale information includes any one or more of the diagonal length, width and height of the target roller detection frame. If the current frame image does not meet the qualification conditions, the inspection robot is adjusted. The qualification conditions mean that the center point coordinates of the target roller detection frame in the current frame image are within the center point tolerance range corresponding to the target viewpoint, and the scale information of the target roller detection frame in the current frame image is within the scale tolerance range corresponding to the target viewpoint. After the inspection robot is adjusted, the center point coordinates and scale information of the target roller detection frame corresponding to the target viewpoint in the current frame image collected by the inspection robot are repeatedly obtained until the qualified conditions are met. After the qualification conditions are met, the current horizontal angle and current pitch angle of the gimbal, the current height of the lifting arm, and the current focal length of the inspection camera are recorded in the work order to be configured, and the inspection work order corresponding to the inspection point is obtained.

[0009] Secondly, embodiments of the present invention provide a roller inspection work order configuration device, applied to an inspection robot, wherein the inspection robot includes an inspection camera deployed on a gimbal, the gimbal being mounted on a lifting arm, and the device includes: The first processing unit is used to obtain the center point coordinates and scale information of the target roller detection frame corresponding to the target viewpoint in the current frame image collected by the inspection robot when the inspection robot travels to the preset inspection point. The scale information includes any one or more of the diagonal length, width and height of the target roller detection frame. The first processing unit is further configured to adjust the inspection robot if the current frame image does not meet the qualification conditions; the qualification conditions refer to the fact that the center point coordinates of the target idler roller detection frame in the current frame image are within the center point tolerance range corresponding to the target viewpoint, and the scale information of the target idler roller detection frame in the current frame image is within the scale tolerance range corresponding to the target viewpoint. The first processing unit is also used to repeatedly acquire the center point coordinates and scale information of the target roller detection frame corresponding to the target viewpoint in the current frame image collected by the inspection robot after the inspection robot has been adjusted, until the qualified conditions are met. The second processing unit is used to record the current horizontal angle and current pitch angle of the gimbal, the current height of the lifting arm, and the current focal length of the inspection camera into the work order to be configured after the qualification conditions are met, so as to obtain the inspection work order corresponding to the inspection point.

[0010] Thirdly, embodiments of the present invention provide a storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method.

[0011] Fourthly, embodiments of the present invention provide an electronic device, the electronic device comprising: a processor and a memory, the memory being used to store one or more programs; when the one or more programs are executed by the processor, the above-described method is implemented.

[0012] Compared to existing technologies, the present invention provides a method, apparatus, storage medium, and device for configuring idler roller inspection work orders. This method acquires the center point coordinates and scale information of the target idler roller detection frame corresponding to the target viewpoint in the current frame image captured by the inspection robot. If the current frame image does not meet the qualification conditions, the inspection robot is adjusted. After meeting the qualification conditions, the current horizontal angle, current pitch angle, current height of the lifting arm, and current focal length of the inspection camera are recorded in the work order to be configured, thus obtaining the inspection work order corresponding to the inspection point. By using the qualification conditions as constraints to adjust the horizontal angle, pitch angle, height of the lifting arm, and focal length of the inspection camera, not only are all mechanical and track deformation errors absorbed and digested, but smooth and stable gimbal compensation is also achieved under variable focal length conditions. This ensures that the images captured by the inspection robot for each idler roller have a highly consistent composition standard, greatly improving the data reliability of subsequent secondary analysis algorithms such as infrared abnormal temperature measurement and voiceprint abnormality recognition, while reducing manual intervention and improving configuration efficiency.

[0013] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0014] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.

[0016] Figure 2 This is one of the flowcharts illustrating the idler roller inspection work order configuration method provided in this embodiment of the invention.

[0017] Figure 3 This is the second flowchart illustrating the idler roller inspection work order configuration method provided in this embodiment of the invention.

[0018] Figure 4 This is the third flowchart illustrating the idler roller inspection work order configuration method provided in this embodiment of the invention.

[0019] Figure 5 This is a schematic diagram of a unit for configuring a roller inspection work order according to an embodiment of the present invention.

[0020] In the diagram: 10-Processor; 11-Memory; 12-Bus; 13-Communication interface; 501-First processing unit; 502-Second processing unit. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0022] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0023] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this invention, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0024] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0025] In the description of this invention, it should be noted that the terms "upper," "lower," "inner," "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship in which the product of this invention is usually placed when in use. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limiting this invention.

[0026] In the description of this invention, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set" and "connection" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0027] The following detailed description of some embodiments of the present invention is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0028] This invention provides an electronic device, which can be a control device for an inspection robot, or a mobile phone, computer, or server device that is communicatively connected to the control device. The inspection robot includes an inspection camera deployed on a gimbal, which is mounted on a lifting arm. Please refer to... Figure 1 This is a schematic diagram of the structure of an electronic device. The electronic device includes a processor 10, a memory 11, and a bus 12. The processor 10 and the memory 11 are connected via the bus 12. The processor 10 is used to execute executable modules, such as computer programs, stored in the memory 11.

[0029] Processor 10 can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the idler roller inspection work order configuration method can be completed through the integrated logic circuits in the hardware of processor 10 or through software instructions. The aforementioned processor 10 can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0030] The memory 11 may include high-speed random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage.

[0031] Bus 12 can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. Figure 1 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus 12 or one type of bus 12.

[0032] The memory 11 is used to store programs, such as the program corresponding to the idler inspection work order configuration device. The idler inspection work order configuration device includes at least one software function module that can be stored in the memory 11 in the form of software or firmware or embedded in the operating system (OS) of the electronic device. After receiving the execution instruction, the processor 10 executes the program to implement the idler inspection work order configuration method.

[0033] The electronic device provided in this embodiment of the invention may further include a communication interface 13. The communication interface 13 is connected to the processor 10 via a bus.

[0034] It should be understood that, Figure 1 The structure shown is only a partial schematic diagram of the electronic device; the electronic device may also include components that are larger than... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown. Figure 1 The components shown can be implemented using hardware, software, or a combination thereof.

[0035] The method for configuring idler roller inspection work orders provided in this embodiment of the invention can be applied to, but is not limited to, inspection robots. Figure 1 For the specific process of the electronic devices shown, please refer to [link / reference]. Figure 2 The configuration methods for idler roller inspection work orders include: S23, S24, S25 and S28, which are described in detail below.

[0036] S23, obtain the center point coordinates and scale information of the target roller detection frame corresponding to the target perspective in the current frame image collected by the inspection robot (which may be the latest frame in the high frame rate video stream collected by the inspection robot).

[0037] The scale information includes one or more of the diagonal length, width, and height of the target roller detection frame, and the current frame image can be the latest frame in the high frame rate video stream collected by the inspection robot.

[0038] When the current mileage reaches the preset mileage, the inspection robot can be determined to travel to the preset inspection point, and S23 will be executed at this time.

[0039] A lightweight idler detection module (such as an edge-deployed YOLO model) can be used to extract the target idler detection box corresponding to the target viewpoint in the current frame image, thereby obtaining the center point coordinates and scale information of the target idler detection box corresponding to the target viewpoint. Alternatively, an instance segmentation network (such as the Mask R-CNN architecture) can be used to extract the target roller detection box corresponding to the target viewpoint in the current frame image, thereby obtaining the center point coordinates and scale information of the target roller detection box corresponding to the target viewpoint. The instance segmentation network can remove background clutter and output a roller edge mask accurate to the pixel level. The geometric centroid of the mask region can be calculated as the center point to replace the center of the target detection box. This approach can provide more stable feature reference coordinates that are unaffected by occlusion in extreme environments with heavy dust occlusion.

[0040] S24. Determine whether the current frame image meets the qualification conditions. If not, proceed to S15; if yes, proceed to S28.

[0041] The qualified condition means that the center point coordinates of the target idler detection box in the current frame image are within the center point tolerance range corresponding to the target viewpoint, and the scale information of the target idler detection box in the current frame image is within the scale tolerance range corresponding to the target viewpoint. The scale tolerance range can refer to any one or more of the width tolerance range, height tolerance range, and diagonal length tolerance range of the detection box.

[0042] S25, adjust the inspection robot.

[0043] After the inspection robot is adjusted, S23 is executed repeatedly to obtain the center point coordinates and scale information of the target roller detection frame corresponding to the target perspective in the current frame image collected by the inspection robot (which is the video frame image re-collected after the inspection robot is adjusted) until the qualified conditions are met.

[0044] S28 records the current horizontal angle and current pitch angle of the gimbal, the current height of the lifting arm, and the current focal length of the inspection camera into the work order to be configured, thus obtaining the inspection work order corresponding to the inspection point.

[0045] In the roller inspection work order configuration provided in this embodiment of the invention, the horizontal angle, pitch angle, height of the lifting arm, and focal length of the inspection camera are adjusted with the qualification conditions as constraints. This not only absorbs and digests all mechanical errors and track deformation errors, but also achieves smooth and stable gimbal compensation under variable focal length conditions. This ensures that the images captured by the inspection robot for each roller have a highly consistent composition standard, which greatly improves the data reliability of subsequent secondary analysis algorithms such as infrared abnormal temperature measurement and voiceprint abnormality recognition, and reduces human intervention, thereby improving configuration efficiency.

[0046] Optionally, S25, adjustments are made to the inspection robot, including S251 and S252, which are described in detail below.

[0047] S251, based on the center point coordinates of the target idler roller detection frame in the current frame image and the reference center coordinates under the target viewpoint, adjust one or more of the following: the horizontal angle of the gimbal, the pitch angle of the gimbal, and the height of the lifting arm. S252, adjust the focal length of the inspection camera based on the scale information of the target roller detection frame in the current frame image and the reference scale information under the target view.

[0048] Optionally, S251, based on the center point coordinates of the target idler roller detection frame in the current frame image and the reference center coordinates under the target viewpoint, adjust one or more of the following: the horizontal angle of the gimbal, the pitch angle of the gimbal, and the height of the lifting arm, including: S2511 and S2512, which are described in detail below.

[0049] S2511, the deviation between the center point coordinates of the target roller detection box in the current frame image and the reference center coordinates under the target view is taken as the pixel-level deviation.

[0050] S2512 adjusts one or more of the following based on pixel-level deviation: the horizontal angle of the gimbal, the pitch angle of the gimbal, and the height of the lifting arm.

[0051] Optionally, the pixel-level deviation includes horizontal pixel deviation (the horizontal Euclidean distance between the center point coordinates of the target roller detection frame in the current frame image and the reference center coordinates under the target view) and vertical pixel deviation (the vertical Euclidean distance between the center point coordinates of the target roller detection frame in the current frame image and the reference center coordinates under the target view), S2512, adjusting any one or more of the pan-tilt head's horizontal angle, pan-tilt head's pitch angle, and lifting arm height according to the pixel-level deviation, including: S2512A1 and S2512A2, which are specifically described below.

[0052] S2512A1 converts pixel-level deviations into fine-tuning instructions for the gimbal and / or lifting arm, including: converting horizontal pixel deviations into fine-tuning instructions for the gimbal's horizontal angle (Yaw), and converting vertical pixel deviations into fine-tuning instructions for the gimbal's pitch angle (Pitch) or the lifting arm's height.

[0053] Pixel-level deviations can be converted into fine-tuning commands for the gimbal and / or lifting arm using coordinate transformation matrices and inverse kinematics models.

[0054] S2512A2 controls the pan-tilt unit and / or lifting arm to execute corresponding fine-tuning commands, thereby adjusting one or more of the pan-tilt unit's horizontal angle, pan-tilt unit's pitch angle, and lifting arm's height.

[0055] Optionally, the pixel-level deviation includes horizontal pixel deviation and vertical pixel deviation. S2512, adjusts one or more of the pan-tilt head's horizontal angle, pan-tilt head's pitch angle, and lifting arm height according to the pixel-level deviation, including: S2512B1-S2512B4, as specifically described below.

[0056] S2512B1 generates a gimbal horizontal angle adjustment prompt based on the horizontal pixel deviation.

[0057] The gimbal horizontal angle adjustment prompt includes the direction of horizontal angle adjustment (e.g., left or right), and optionally, the adjustment range of the horizontal angle.

[0058] S2512B2 displays gimbal horizontal angle adjustment prompts on the AR rendering interface corresponding to the inspection point, so that staff can adjust the gimbal horizontal angle according to the prompts.

[0059] S2512B3 generates gimbal tilt angle adjustment prompts based on vertical pixel deviation.

[0060] The gimbal tilt angle adjustment prompt includes the direction of the tilt angle adjustment (e.g., up or down), and optionally, the adjustment range of the tilt angle.

[0061] S2512B4 displays gimbal tilt angle adjustment prompts on the AR rendering interface corresponding to the inspection point, so that staff can adjust the gimbal tilt angle according to the prompts.

[0062] Optionally, S252, the focal length of the inspection camera is adjusted according to the scale information of the target roller detection box in the current frame image and the reference scale information under the target viewpoint, including: S2521 and S2522.

[0063] S2521, the deviation between the scale information of the target roller detection box in the current frame image and the reference scale information under the target viewpoint is taken as the scale deviation; S2522, adjusts the focal length of the inspection camera according to the scale deviation.

[0064] Optionally, S2522 adjusts the focal length of the inspection camera according to the scale deviation, including: S2522A1 and S2522A2.

[0065] S2522A1 converts scale deviation into focal length scaling instructions for inspection cameras.

[0066] Scale deviations can be converted into focal length scaling commands for inspection cameras using coordinate transformation matrices and inverse kinematics models.

[0067] S2522A2 controls the inspection camera to execute the corresponding focal length scaling command, thereby adjusting the focal length of the inspection camera.

[0068] Optionally, S2522 adjusts the focal length of the inspection camera according to the scale deviation, including: S2522B1 and S2522B2.

[0069] S2522B1 generates camera focal length scaling prompts based on scale deviation.

[0070] The camera focal length zoom prompt includes the direction of focal length adjustment (e.g., zoom in or zoom out), and optionally, the magnitude of focal length adjustment.

[0071] S2522B2 displays camera focal length scaling prompts on the AR rendering interface corresponding to the inspection point, so that staff can adjust the focal length of the inspection camera according to the camera focal length scaling prompts.

[0072] In the idler roller inspection work order configuration method provided in this embodiment of the invention, relevant prompts can be displayed on the AR rendering interface corresponding to the inspection point, transforming the highly technical and tedious configuration work into a visually guided process that can be operated by all personnel, greatly reducing project implementation costs. The initial work order point exploration and registration work, which originally required senior field engineers to spend several weeks, has been reshaped into a goal-oriented assembly line operation through the visual gesture guidance method.

[0073] Furthermore, the AR rendering interface can display the reference center coordinates, reference scale information, center point tolerance range, and scale tolerance range from the target's perspective, as well as the center coordinates and scale information of the target roller detection box in the current frame image. Through continuously feedback and correction text prompts, a WYSIWYG operating environment is constructed.

[0074] Please refer to Figure 3 When the inspection robot travels to the preset inspection point, the roller inspection work order configuration method also includes: S21 and S22, as follows.

[0075] S21, when the inspection robot travels to the preset inspection point, control the inspection robot to stop moving and load the work order to be configured for the inspection point.

[0076] The work order to be configured includes the target viewpoint and the preset gimbal angle baseline value; S22, adjusts the horizontal and / or pitch angles of the gimbal based on the gimbal angle baseline value.

[0077] Optionally, the current horizontal angle of the pan-tilt unit (PTZ) can be obtained. The PTZ angle baseline value is a built-in 175-degree baseline, and the four monitoring directions are set as +40, +125, -50, and -140. The difference between the current horizontal angle and the angle scale of the target monitoring direction is determined. The angle scale difference is dynamically displayed, and a text prompt is given, such as Li Rui: "The horizontal angle is still XX degrees away, please continue to adjust to XX degrees." If the adjustment is not completed within a certain time, the prompt is repeated every 5 seconds.

[0078] It should be noted that the above prompts can also be converted into voice prompts and broadcast accordingly.

[0079] Regarding how to determine the center point tolerance range and scale tolerance range corresponding to the target viewpoint, this embodiment of the invention also provides an optional implementation method, please refer to the following. The method for configuring idler roller inspection work orders includes: S11, obtain inspection samples from different perspectives (such as the upper perspective near end and the middle perspective near end).

[0080] The inspection criteria include: the number of samples must exceed a set threshold; the proportion of images occupied by idlers must exceed a threshold; and the clarity of the idler area in the image must be higher than a set clarity threshold. Different types of idlers are monitored from different viewing angles. For example, a top-viewing angle focuses on the far-end idlers, while a mid-viewing angle focuses on the near-end idlers. Therefore, the type of target idler corresponds to the viewing angle.

[0081] S12, extract the four-dimensional statistical feature vector of the target idler roller detection frame in each inspection sample under each view, and then construct the historical inspection feature set under each view.

[0082] The four-dimensional statistical feature vector includes the horizontal coordinate (pixels) of the center point of the target idler detection frame, the vertical coordinate (pixels) of the center point of the target idler detection frame, the width (pixels) of the target idler detection frame, and the height (pixels) of the target idler detection frame. The historical inspection feature set includes the four-dimensional statistical feature vector of the target idler detection frame in each inspection sample under the corresponding viewpoint.

[0083] The expression for the historical inspection feature set is:

[0084] in, This represents the set of historical inspection features from the j-th perspective. Let represent the four-dimensional statistical feature vector of the i-th inspection sample from the j-th perspective. Let x represent the x-coordinate of the center point of the detection frame of the target idler roller of the i-th inspection sample from the j-th viewpoint. The ordinate represents the center point of the detection frame of the target idler roller for the i-th inspection sample from the j-th viewpoint. Let represent the width of the target idler roller detection frame for the i-th inspection sample from the j-th viewpoint. Let J represent the height of the target idler roller detection frame of the i-th inspection sample from the j-th viewpoint, 1≤i≤I, 1≤j≤J, where I represents the total number of inspection samples from the j-th viewpoint and J represents the total number of viewpoints.

[0085] S13, input the set of historical inspection features under each viewpoint into the K-component Gaussian mixture model to obtain the probability distribution density of the target roller detection box in the two-dimensional pixel coordinate system in each inspection sample under each viewpoint.

[0086] The formula for the probability density function is:

[0087] in, Let represent the probability distribution density of the i-th inspection sample from the j-th perspective. Let represent the four-dimensional statistical feature vector of the i-th inspection sample from the j-th perspective. The mixing coefficient of the k-th Gaussian component under the j-th viewpoint ( ), Let represent the mean vector of the historical inspection feature set from the j-th perspective. Let the covariance matrix of the historical inspection feature set from the j-th perspective be denoted as . Let K represent the multivariate Gaussian distribution of the historical inspection feature set from the j-th perspective, and let K represent the preset number of components. The number of components K is selected using the Bayesian Information Criterion (BIC) to avoid overfitting.

[0088] S14, take the inspection sample with the highest probability distribution density under the j-th view as the benchmark inspection sample under the j-th view, take the center point coordinates of the benchmark inspection sample under the j-th view as the benchmark center coordinates under the j-th view, and take the scale information of the benchmark inspection sample under the j-th view as the benchmark scale information under the j-th view.

[0089] S15. Determine the center point tolerance radius corresponding to the j-th viewpoint based on the joint standard deviation of the center point coordinates of the target idler roller detection frame in all inspection samples under the j-th viewpoint. Determine the center point tolerance range based on the reference center coordinates and the center point tolerance radius under the j-th viewpoint.

[0090] The center point tolerance radius is , Let represent the joint standard deviation of the center point coordinates (including the x and y coordinates) of the target idler roller detection frame in all inspection samples under the j-th viewpoint, centered on the center point coordinates (including the x and y coordinates) of the benchmark inspection sample under the j-th viewpoint. The standard deviation of the X and Y axes may not be the same, as the radius of tolerance is the center point, thus forming an elliptical region. Let represent the four-dimensional statistical feature vector of the benchmark inspection sample from the j-th perspective.

[0091] S16. Determine the scale tolerance corresponding to the j-th viewpoint based on the joint standard deviation of the scale information of the target idler detection frame in all inspection samples under the j-th viewpoint, and determine the scale tolerance range based on the benchmark scale information and scale tolerance under the j-th viewpoint.

[0092] When the scale information is the width of the target idler roller detection frame, the scale tolerance range is: ,in, Let be the joint standard deviation of the width of the target idler roller detection frame in all inspection samples from the j-th viewpoint. This represents the width tolerance corresponding to the j-th viewpoint; When the scale information is the height of the target idler roller detection frame, the scale tolerance range is: ,in, Let be the joint standard deviation of the height of the target idler roller detection frame in all inspection samples from the j-th viewpoint. The height tolerance corresponds to the j-th viewpoint; This tolerance range ensures that approximately 95% of historical high-quality samples fall within the reference mapping area, providing both a clear closed target boundary and reasonable mapping flexibility.

[0093] To address the pseudo-steady-state problem, this embodiment of the invention also provides an optional implementation method, please refer to... Figure 4 After meeting the qualification conditions, the configuration method for idler roller inspection work orders also includes: S26 and S27, which are described in detail below.

[0094] S26, acquire the target image collected by the inspection robot.

[0095] The target image is a video frame image with a detection confidence score greater than a preset confidence score threshold. The detection confidence score reflects the certainty of the target idler roller detection frame recognition.

[0096] S27. Determine the temporal stability score of the target image based on the detection confidence, center point jitter amplitude, frame drop ratio, and size change rate of the target image.

[0097] Optionally, the detection confidence, center point jitter amplitude, frame fall ratio, and size change rate of the target image are used as vector inputs into the weighted scoring model to output the temporal stability score of the target image.

[0098] When the number of consecutive occurrences of target images that have reached the set safety threshold in terms of temporal stability score exceeds the set value, false signals within the oscillation cycle are perfectly shielded. S28 can be executed to record the current horizontal angle, current pitch angle of the gimbal, current height of the lifting arm, and current focal length of the inspection camera into the work order to be configured, thus obtaining the inspection work order corresponding to the inspection point.

[0099] It should be understood that the inspection robot can be adjusted when the timing stability score does not reach the set safety threshold.

[0100] By setting a condition that the number of consecutive occurrences of target images reaching a set safety threshold in terms of temporal stability score exceeds a set value, the system significantly enhances the temporal anti-interference capability of data registration under harsh working conditions, ensuring the purity of the original coordinate library. To address the high-frequency vibrations caused by the high-speed operation of the conveyor belt and the mechanical aftershocks from operation, a multi-frame temporal stability confirmation mechanism is introduced. By comprehensively examining four dimensions—detection confidence, center point jitter amplitude, frame drop ratio, and size change rate—the system effectively isolates all transient jitter images and incompletely focused images. This reduces the possibility of false coordinates being written into the work order database, greatly improving the success rate of the robot reusing historical coordinates in the future, and ensuring that the entire autonomous control logic is built on an absolutely reliable reference benchmark.

[0101] Center Jitter Amplitude: Calculates the absolute value of the displacement of the center point coordinates between the current target image and the target image of the previous frame, reflecting the degree of mechanical vibration.

[0102] Intersection over Reference Area Ratio (IOU): Calculates the intersection-over-union ratio (IOU) between the actual target roller detection box in the target image and the ideal reference box (the target roller detection box in the benchmark inspection sample under the corresponding viewpoint) to evaluate the composition fit.

[0103] Size Change Rate: The scaling ratio of the target roller detection box between the current target image and the target image in the previous frame, used to assess whether the camera's autofocus and zoom mechanism has completely stopped and converged.

[0104] In one alternative implementation, a Kalman filter mechanism or a nonlinear extended Kalman filter (EKF) can be used to track the state of the acquired two-dimensional coordinates of the idler roller. The center coordinates and deviations observed in each frame are considered as inputs with observation noise. When the prior covariance matrix output by the Kalman filter converges to a certain minimum threshold, which mathematically represents a steady state of dynamic equilibrium between the target trajectory and the observation system, the data can be safely saved and confirmed.

[0105] Please see Figure 5 , Figure 5 The present invention provides a roller inspection work order configuration device, which is optionally applied to the electronic equipment described above.

[0106] The idler roller inspection work order configuration device is applied to the inspection robot, which includes an inspection camera deployed on a gimbal and the gimbal is mounted on a lifting arm. The idler roller inspection work order configuration device includes a first processing unit 501 and a second processing unit 502.

[0107] The first processing unit 501 is used to obtain the center point coordinates and scale information of the target roller detection frame corresponding to the target viewpoint in the current frame image collected by the inspection robot when the inspection robot travels to the preset inspection point. The scale information includes any one or more of the diagonal length, width and height of the target roller detection frame. The first processing unit 501 is also used to adjust the inspection robot if the current frame image does not meet the qualification conditions; the qualification conditions refer to the fact that the center point coordinates of the target roller detection box in the current frame image are within the center point tolerance range corresponding to the target viewpoint, and the scale information of the target roller detection box in the current frame image is within the scale tolerance range corresponding to the target viewpoint. The first processing unit 501 is also used to repeatedly acquire the center point coordinates and scale information of the target roller detection frame corresponding to the target viewpoint in the current frame image collected by the inspection robot after the inspection robot has been adjusted, until the qualified conditions are met. The second processing unit 502 is used to record the current horizontal angle, current pitch angle of the gimbal, current height of the lifting arm, and current focal length of the inspection camera into the work order to be configured after the qualification conditions are met, so as to obtain the inspection work order corresponding to the inspection point.

[0108] The second processing unit 502 can execute S28 as described above, and the first processing unit 501 can execute other steps in the above method embodiment.

[0109] It should be noted that the idler roller inspection work order configuration device provided in this embodiment can execute the method flow shown in the above-described method flow embodiment to achieve the corresponding technical effects. For the sake of brevity, any parts not mentioned in this embodiment can be referred to the corresponding content in the above-described embodiments.

[0110] This invention also provides a storage medium storing computer instructions and programs, which, when read and executed, perform the idler roller inspection work order configuration method described above. The storage medium may include memory, flash memory, registers, or a combination thereof.

[0111] The following provides an electronic device, which can be a control device for an inspection robot, or a mobile phone, computer, or server device that communicates with the control device. This electronic device, for example... Figure 1 As shown, the above-described idler inspection work order configuration method can be implemented. Specifically, the electronic device includes: a processor 10, a memory 11, and a bus 12. The processor 10 can be a CPU. The memory 11 is used to store one or more programs, and when one or more programs are executed by the processor 10, the idler inspection work order configuration method of the above embodiment is executed.

[0112] In summary, the present invention provides a method, apparatus, storage medium, and device for configuring idler inspection work orders. This involves acquiring the center point coordinates and scale information of the target idler detection frame corresponding to the target viewpoint in the current frame image captured by the inspection robot. If the current frame image does not meet the qualification conditions, the inspection robot is adjusted. After meeting the qualification conditions, the current horizontal angle, current pitch angle, current height of the lifting arm, and current focal length of the inspection camera are recorded in the work order to be configured, thus obtaining the inspection work order corresponding to the inspection point. By using the qualification conditions as constraints to adjust the horizontal angle, pitch angle, height of the lifting arm, and focal length of the inspection camera, not only are all mechanical and track deformation errors absorbed and digested, but smooth and stable gimbal compensation is also achieved under variable focal length conditions. This ensures that the images captured by the inspection robot for each idler have a highly consistent composition standard, greatly improving the data reliability of subsequent secondary analysis algorithms such as infrared abnormal temperature measurement and voiceprint abnormality recognition, while reducing manual intervention and improving configuration efficiency.

[0113] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0114] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A method for configuring idler roller inspection work orders, characterized in that, An inspection robot, comprising an inspection camera deployed on a gimbal, the gimbal being mounted on a lifting arm, wherein when the inspection robot travels to a preset inspection point, the method includes: The coordinates and scale information of the center point of the target roller detection frame corresponding to the target viewpoint in the current frame image collected by the inspection robot are obtained, wherein the scale information includes any one or more of the diagonal length, width and height of the target roller detection frame. If the current frame image does not meet the qualification conditions, the inspection robot is adjusted. The qualification conditions mean that the center point coordinates of the target roller detection frame in the current frame image are within the center point tolerance range corresponding to the target viewpoint, and the scale information of the target roller detection frame in the current frame image is within the scale tolerance range corresponding to the target viewpoint. After the inspection robot is adjusted, the center point coordinates and scale information of the target roller detection frame corresponding to the target viewpoint in the current frame image collected by the inspection robot are repeatedly obtained until the qualified conditions are met. After the qualification conditions are met, the current horizontal angle and current pitch angle of the gimbal, the current height of the lifting arm, and the current focal length of the inspection camera are recorded in the work order to be configured, and the inspection work order corresponding to the inspection point is obtained.

2. The method for configuring idler roller inspection work orders as described in claim 1, characterized in that, The adjustment of the inspection robot includes: Based on the center point coordinates of the target idler roller detection frame in the current frame image and the reference center coordinates from the target viewpoint, adjust one or more of the following: the horizontal angle of the gimbal, the pitch angle of the gimbal, and the height of the lifting arm. The focal length of the inspection camera is adjusted based on the scale information of the target roller detection frame in the current frame image and the reference scale information from the target viewpoint.

3. The method for configuring idler roller inspection work orders as described in claim 2, characterized in that, The step of adjusting one or more of the following based on the center point coordinates of the target idler detection frame in the current frame image and the reference center coordinates from the target viewpoint: This includes: The deviation between the center point coordinates of the target roller detection box in the current frame image and the reference center coordinates under the target viewpoint is taken as the pixel-level deviation; Adjust one or more of the following based on the pixel-level deviation: the horizontal angle of the gimbal, the pitch angle of the gimbal, and the height of the lifting arm.

4. The method for configuring idler roller inspection work orders as described in claim 3, characterized in that, The pixel-level deviation includes horizontal pixel deviation and vertical pixel deviation. Adjusting one or more of the following based on the pixel-level deviation: the horizontal angle of the gimbal, the pitch angle of the gimbal, and the height of the lifting arm: Converting the pixel-level deviation into fine-tuning instructions for the gimbal and / or lifting arm includes: converting the horizontal pixel deviation into fine-tuning instructions for the horizontal angle of the gimbal, and converting the vertical pixel deviation into fine-tuning instructions for the pitch angle of the gimbal or the height of the lifting arm. Control the gimbal and / or lifting arm to execute corresponding fine-tuning commands, thereby adjusting one or more of the following: the horizontal angle of the gimbal, the pitch angle of the gimbal, and the height of the lifting arm.

5. The method for configuring idler roller inspection work orders as described in claim 3, characterized in that, The pixel-level deviation includes horizontal pixel deviation and vertical pixel deviation. Adjusting one or more of the following based on the pixel-level deviation: the horizontal angle of the gimbal, the pitch angle of the gimbal, and the height of the lifting arm: Based on the horizontal pixel deviation, a gimbal horizontal angle adjustment prompt is generated, wherein the gimbal horizontal angle adjustment prompt includes the direction of horizontal angle adjustment; The gimbal horizontal angle adjustment prompt is displayed on the AR rendering interface corresponding to the inspection point; Based on the vertical pixel deviation, a gimbal pitch angle adjustment prompt is generated, wherein the gimbal pitch angle adjustment prompt includes the adjustment direction of the pitch angle; The gimbal tilt angle adjustment prompt is displayed on the AR rendering interface corresponding to the inspection point.

6. The method for configuring idler roller inspection work orders as described in claim 2, characterized in that, The step of adjusting the focal length of the inspection camera based on the scale information of the target roller detection frame in the current frame image and the reference scale information under the target viewpoint includes: The deviation between the scale information of the target roller detection box in the current frame image and the reference scale information under the target viewpoint is taken as the scale deviation; The focal length of the inspection camera is adjusted according to the scale deviation.

7. The method for configuring idler roller inspection work orders as described in claim 6, characterized in that, The step of adjusting the focal length of the inspection camera according to the scale deviation includes: The scale deviation is converted into a focal length scaling command for the inspection camera; The inspection camera is controlled to execute the corresponding focal length scaling command, thereby adjusting the focal length of the inspection camera.

8. The method for configuring idler roller inspection work orders as described in claim 6, characterized in that, The step of adjusting the focal length of the inspection camera according to the scale deviation includes: Based on the scale deviation, a camera focal length scaling prompt is generated, wherein the camera focal length scaling prompt includes the direction of focal length adjustment; The camera focal length scaling prompt is displayed on the AR rendering interface corresponding to the inspection point.

9. The method for configuring idler roller inspection work orders as described in claim 1, characterized in that, When the inspection robot travels to the preset inspection point, the method further includes: Control the inspection robot to stop moving and load the work order to be configured for the inspection point, wherein the work order to be configured includes the target view and the preset gimbal angle baseline value; The horizontal and / or pitch angles of the gimbal are adjusted based on the gimbal angle baseline value.

10. The method for configuring idler roller inspection work orders as described in claim 1, characterized in that, The method includes: Obtain inspection samples from different perspectives. The inspection samples are historical inspection images in which the proportion of the image occupied by the idler roller exceeds the proportion threshold and the clarity of the idler roller area in the image is higher than the set clarity threshold. Extract the four-dimensional statistical feature vector of the target idler detection frame in each inspection sample under each viewpoint, and then construct the historical inspection feature set under each viewpoint. The four-dimensional statistical feature vector includes the horizontal coordinate of the center point of the target idler detection frame, the vertical coordinate of the center point of the target idler detection frame, the width of the target idler detection frame, and the height of the target idler detection frame. The historical inspection feature set includes the four-dimensional statistical feature vector of the target idler detection frame in each inspection sample under the corresponding viewpoint. The set of historical inspection features under each viewpoint is substituted into a Gaussian mixture model with K components to obtain the probability distribution density of the target roller detection box in the two-dimensional pixel coordinate system in each inspection sample under each viewpoint. The inspection sample with the highest probability distribution density under the j-th viewpoint is taken as the benchmark inspection sample under the j-th viewpoint, the center point coordinates of the benchmark inspection sample under the j-th viewpoint are taken as the benchmark center coordinates under the j-th viewpoint, and the scale information of the benchmark inspection sample under the j-th viewpoint is taken as the benchmark scale information under the j-th viewpoint. The center point tolerance radius corresponding to the j-th viewpoint is determined by the joint standard deviation of the center point coordinates of the target idler roller detection frame in all inspection samples under the j-th viewpoint, and the center point tolerance range is determined by the reference center coordinates and the center point tolerance radius under the j-th viewpoint. The scale tolerance corresponding to the j-th viewpoint is determined by the joint standard deviation of the scale information of the target idler roller detection frame in all inspection samples under the j-th viewpoint. The scale tolerance range is determined by the baseline scale information and scale tolerance under the j-th viewpoint.

11. The method for configuring idler roller inspection work orders as described in claim 1, characterized in that, After the qualification conditions are met, the method further includes: The target image collected by the inspection robot is obtained, wherein the target image is a video frame image with a detection confidence score greater than a preset confidence score threshold; The temporal stability score of the target image is determined based on the detection confidence, center point jitter amplitude, frame drop ratio, and size change rate of the target image. When the number of consecutive occurrences of a target image whose temporal stability score reaches a set safety threshold exceeds a set value, the current horizontal angle and current pitch angle of the gimbal, the current height of the lifting arm, and the current focal length of the inspection camera are recorded in the work order to be configured, thus obtaining the inspection work order corresponding to the inspection point.

12. A device for configuring inspection work orders for idler rollers, characterized in that, An inspection robot is used in a device that includes an inspection camera deployed on a gimbal, the gimbal being mounted on a lifting arm. The device includes: The first processing unit is used to obtain the center point coordinates and scale information of the target roller detection frame corresponding to the target viewpoint in the current frame image collected by the inspection robot when the inspection robot travels to the preset inspection point. The scale information includes any one or more of the diagonal length, width and height of the target roller detection frame. The first processing unit is further configured to adjust the inspection robot if the current frame image does not meet the qualification conditions; the qualification conditions refer to the fact that the center point coordinates of the target idler roller detection frame in the current frame image are within the center point tolerance range corresponding to the target viewpoint, and the scale information of the target idler roller detection frame in the current frame image is within the scale tolerance range corresponding to the target viewpoint. The first processing unit is also used to repeatedly acquire the center point coordinates and scale information of the target roller detection frame corresponding to the target viewpoint in the current frame image collected by the inspection robot after the inspection robot has been adjusted, until the qualified conditions are met. The second processing unit is used to record the current horizontal angle and current pitch angle of the gimbal, the current height of the lifting arm, and the current focal length of the inspection camera into the work order to be configured after the qualification conditions are met, so as to obtain the inspection work order corresponding to the inspection point.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-11.

14. An electronic device, characterized in that, include: Processor and memory, the memory being used to store one or more programs; When the one or more programs are executed by the processor, the method as described in any one of claims 1-11 is implemented.

Images