A fire alarm host operation panel accurate identification method and system

Abstract

The application discloses a fire alarm host operation panel accurate identification method and system, and belongs to the technical field of intelligent robots. The method comprises the following steps: acquiring a to-be-identified image containing a fire alarm host operation panel; determining position information of the operation panel in a depth direction based on a structure light feature; determining position information of the operation panel in a plane direction based on a macroscopic contour feature; and determining final three-dimensional space coordinates of a target operation area on the operation panel based on the position information in the depth direction and the position information in the plane direction. The application separates the visual feature sources of depth measurement and plane positioning, uses a follow-up structure light to specially solve the depth, and uses an inherent macroscopic contour of the panel to specially solve the plane translation, thereby improving the perspective confusion and positioning reference drift problems of a single visual feature source on a flat and textureless surface.

Description

Technical Field

[0001] This application relates to the fields of computer vision and intelligent robot technology, and in particular to a method and system for accurate identification of the operation panel of a fire alarm host. Background Technology

[0002] In the field of smart fire protection, utilizing rail-mounted inspection robots to automate the status monitoring and emergency operation of fire alarm control panels in fire control rooms is a key technology for improving fire management efficiency and emergency response speed. However, when performing delicate tasks such as button operations, robots face significant challenges in posture calibration. Due to mechanical wear of the rails, installation tolerances, and the cumulative errors of the robot's multi-axis arms, there will inevitably be slight three-dimensional spatial translational and rotational deviations between the robot's end effector and the target panel.

[0003] Existing calibration methods, on the one hand, can provide high-precision positioning by affixing artificial markers such as QR codes to the panel, but this seriously violates the engineering red line that fire protection facilities are not allowed to undergo any physical modifications. On the other hand, using general 3D vision (such as depth cameras) to reconstruct the entire panel results in extremely low reliability of feature point matching algorithms and huge computational resource consumption because the host panel is usually made of smooth metal or plastic, which lacks stable texture features and makes it difficult to meet real-time requirements.

[0004] Therefore, how to achieve low-cost and high-precision 3D pose calibration on flat, textureless surfaces without modifying existing facilities is a long-standing and urgent technical challenge in this field. Summary of the Invention

[0005] To address the aforementioned issues, in a first aspect, this application provides a method for accurate identification of a fire alarm control panel, comprising: acquiring an image to be identified containing a fire alarm control panel, wherein the image to be identified includes a structured light feature formed on the control panel and projected by an asymmetric structured light emitter, and a macroscopic contour feature of the control panel; determining position information of the control panel in a depth direction based on the structured light feature; determining position information of the control panel in a planar direction based on the macroscopic contour feature; and determining the final three-dimensional spatial coordinates of a target operating area on the control panel based on the position information in the depth direction and the position information in the planar direction.

[0006] Optionally, the method, according to any of the preceding claims, is characterized in that the structured light feature is a parallel light feature composed of at least two parallel light bands in physical space; the step of determining the position information of the operation panel in a depth direction based on the structured light feature includes: extracting the pixel spacing between the at least two light bands on a preset scanning axis in the image to be identified; and determining the position information in the depth direction based on the pixel spacing and a preset camera model parameter.

[0007] Optionally, the method, according to any of the preceding claims, is characterized in that the preset scanning axis is the horizontal scanning line where the optical center of the image to be identified is located.

[0008] Optionally, the method, according to any of the preceding claims, is characterized in that the macroscopic contour feature is the contour of a preset physical component on the operation panel; the step of determining the position information of the operation panel in a planar direction based on the macroscopic contour feature includes: identifying the contour of the preset physical component in the image to be identified, and extracting the pixel coordinates of a reference physical origin on the contour; determining the position information in the planar direction based on the pixel coordinates of the reference physical origin, the position information in the depth direction, and a preset camera model parameter.

[0009] Optionally, the method, according to any of the preceding claims, is characterized in that the preset physical component is a liquid crystal display screen, and the reference physical origin is a preset vertex of the outline of the liquid crystal display screen.

[0010] Optionally, the method, according to any of the preceding claims, is characterized in that the step of determining the final three-dimensional spatial coordinates of a target operating area on the operation panel based on the position information in the depth direction and the position information in the planar direction includes: combining the position information in the planar direction with the position information in the depth direction to obtain the three-dimensional spatial coordinates of the reference physical origin in the camera coordinate system; obtaining a preset physical offset vector of the target operating area relative to the reference physical origin; vectorically combining the three-dimensional spatial coordinates of the reference physical origin in the camera coordinate system with the preset physical offset vector to obtain the three-dimensional spatial coordinates of the target operating area in the camera coordinate system; and transforming the three-dimensional spatial coordinates of the target operating area in the camera coordinate system to a robotic arm base coordinate system to obtain the final three-dimensional spatial coordinates.

[0011] Secondly, this application provides a precise identification system for a fire alarm control panel, comprising: an image acquisition module for acquiring an image to be identified containing a fire alarm control panel, wherein the image to be identified includes a structured light feature formed on the control panel and projected by an asymmetric structured light emitter, and a macroscopic contour feature of the control panel; a depth position determination module for determining the position information of the control panel in a depth direction based on the structured light feature; a planar position determination module for determining the position information of the control panel in a planar direction based on the macroscopic contour feature; and a target coordinate synthesis module for determining the final three-dimensional spatial coordinates of a target operating area on the control panel based on the position information in the depth direction and the position information in the planar direction.

[0012] Optionally, the system, according to any of the preceding claims, is characterized in that the structured light feature is a parallel light feature composed of at least two parallel light bands in physical space; the depth position determination module is specifically used to: extract the pixel spacing between the at least two light bands on a preset scanning axis in the image to be identified; and determine the position information in the depth direction based on the pixel spacing and a preset camera model parameter.

[0013] Optionally, the system, according to any of the preceding claims, is characterized in that the macroscopic contour feature is the contour of a preset physical component on the operation panel; the planar position determination module is specifically used to: identify the contour of the preset physical component in the image to be identified, and extract the pixel coordinates of a reference physical origin on the contour; and, based on the pixel coordinates of the reference physical origin, the position information in the depth direction, and a preset camera model parameter, determine the position information in the planar direction.

[0014] Optionally, according to any of the preceding claims, the system is characterized in that the target coordinate synthesis module is specifically configured to: combine the position information in the planar direction with the position information in the depth direction to obtain the three-dimensional spatial coordinates of the reference physical origin in the camera coordinate system; obtain a preset physical offset vector of the target operating area relative to the reference physical origin; perform vector combination of the three-dimensional spatial coordinates of the reference physical origin in the camera coordinate system and the preset physical offset vector to obtain the three-dimensional spatial coordinates of the target operating area in the camera coordinate system; and transform the three-dimensional spatial coordinates of the target operating area in the camera coordinate system to a robotic arm base coordinate system to obtain the final three-dimensional spatial coordinates.

[0015] The beneficial effects of the method and system provided in this application are as follows:

[0016] This application constructs a dual-track decoupled visual positioning framework, decomposing the 3D pose calculation problem into two orthogonal and non-interfering sub-problems. First, it utilizes the characteristics of structured light with fixed physical spacing projected by the robot to specifically calculate the normal depth between the robot's end effector and the target panel. This depth measurement is immune to translational errors in any planar direction, exhibiting extremely high robustness. Second, it uses the inherent, unmodified macroscopic physical contours of the fire alarm control panel itself (such as the edge of an LCD screen) as absolute physical anchor points for establishing a planar coordinate system. Once the depth is accurately determined, the position of this anchor point in the image can be unambiguously inversely mapped to the robot's physical translational deviation in the planar direction. This method of completely separating the ranging and source-finding tasks at the feature level improves the technical challenge of depth and translational perspective confusion in traditional monocular structured light on flat, textureless surfaces. It achieves millimeter-level precise positioning with extremely low computational resource consumption (without 3D point cloud reconstruction), enhancing the reliability and applicability of automated robot operation in non-cooperative environments. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating a method for accurate identification of the operation panel of a fire alarm control panel, provided in an embodiment of this application;

[0019] Figure 2 A hardware and software co-architecture diagram of a fire alarm control panel operation panel accurate identification system provided in this application embodiment;

[0020] Figure 3 This application provides a dual-track decoupled physical inverse mapping data flow diagram for embodiments of the present application.

[0021] Figure 4 Simulation diagram of the features of the depth calculation function provided in the embodiments of this application;

[0022] Figure 5 The simulation diagram showing the depth resolution limit provided in the embodiments of this application. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with the accompanying drawings and specific embodiments. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0024] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0025] In a specific implementation, this method employs a dual-track decoupled physical dimension collapse mechanism. Within a single image frame, it simultaneously utilizes the features of actively projected asymmetric structured light and the inherent macroscopic contour features of the target panel, assigning orthogonal localization tasks to these two types of features. This decomposes the complex 3D pose calibration problem into two independent, algebraically solvable subproblems. This method overcomes the technical contradiction in existing technologies where a single visual feature source (whether passive texture or active structured light) cannot simultaneously and accurately calibrate depth and planar translation on smooth, textureless surfaces. It achieves the beneficial effect of millimeter-level precise localization and automated operation with low computational resource consumption without requiring any physical modification to the target environment.

[0026] This application provides a method for accurate identification of the operation panel of a fire alarm control panel. The method is described below. Please refer to the following... Figure 1 , Figure 1 A flowchart illustrating a method for accurate identification of the operation panel of a fire alarm control panel, provided in an embodiment of this application. Figure 1 As shown, in one embodiment, the method is executed in a physical system consisting of a sliding robot, a sensing module (including a near-infrared camera and an asymmetric structured light emitter) fixed to the end of the robot, and an edge domain controller deployed on the robot chassis.

[0027] S100: Acquire an image to be identified that includes a fire alarm control panel, wherein the image to be identified includes a structured light feature formed on the control panel and projected by an asymmetric structured light emitter, and a macroscopic contour feature of the control panel.

[0028] This step aims to capture all the raw information needed for subsequent 3D pose calculation in one go through a sensing operation that integrates active light source and passive imaging.

[0029] Specifically, this step begins with the robot system receiving a top-level instruction to operate a specific fire alarm control panel. The macro-addressing drive engine within the robot chassis, such as a motion controller based on a programmable gate array (PLG), invokes its internally stored fire control room equipment topology map to resolve the target control panel's logical number, such as control panel 03, into specific physical coordinates on a sliding track. Subsequently, this engine drives the track stepper motor, controlling the robot to move along a predetermined track and eventually dock in a pre-defined working area in front of the target control panel. This docking position is a rough macro-position, theoretically, 200 mm from the panel's normal distance. After macro-docking, an edge domain controller deployed on the robot chassis, such as an embedded computing unit integrating an ARM Cortex-A series main control chip and a digital signal processor (DSP), sends an image acquisition trigger command to the sensing module fixed to the end flange of the six-axis robotic arm.

[0030] The sensing module includes an asymmetric structured light emitter and a near-infrared global shutter camera. The asymmetric structured light emitter, for example, a module composed of a laser diode and a custom-designed diffractive optical element (DOE), emits infrared laser light of a specific wavelength (e.g., 850 nm) upon receiving a trigger signal and projects it onto the surface of the fire alarm control panel. The structured light is not a simple point or line, but is designed to have structured light features with specific geometric constraints.

[0031] In some embodiments, the structured light feature projected by the transmitter consists of a main light strip and a secondary light strip, which are parallel to each other in physical space and perpendicular to a preset reference horizontal plane. In one specific implementation, the linewidth of the main light strip is set to 4.0 mm, the linewidth of the secondary light strip is set to 2.0 mm, and the physical distance Dl between the main light strip and the secondary light strip is calibrated to 25.0 mm. This asymmetric linewidth configuration allows an image processing module to uniquely determine the left-right relative position of the structured light feature in the local coordinate system based on the quantized difference in pixel width, eliminating coordinate system mirror ambiguity caused by the robotic arm undergoing a 180-degree tilt.

[0032] In a preferred embodiment, the structured light consists of at least two strictly parallel vertical light bands in physical space, the physical spacing between which is calibrated at the factory and is a known fixed value. Simultaneously, a near-infrared camera, rigidly coplanar with the emitter, performs exposure synchronously to capture a frame of the image to be identified. A narrow-band filter of the corresponding wavelength is mounted in front of the camera lens to minimize ambient visible light interference.

[0033] Due to the presence of narrowband filters and the infrared characteristics of structured light, the image to be identified exhibits unique visual characteristics: on the one hand, the projected structured light features form high-brightness, fine, filamentary lines on the image; on the other hand, since most fire alarm control panels contain a liquid crystal display (LCD), the glass and polarizer materials of the LCD screen exhibit extremely high absorption rates at the 850 nm wavelength (appearing as deep black), while the surrounding metal or plastic panels exhibit strong diffuse reflection of infrared light (appearing as light gray). This difference in the physical response of the materials to specific wavelengths of light causes the rectangular outline of the LCD screen to form a very clear and high-contrast edge in the infrared image, which is the aforementioned macroscopic outline feature.

[0034] Ultimately, this frame contains a digital image matrix that includes both actively projected microscopic geometric information (structured light features) for precise measurement and the physical boundary information (macroscopic contour features) inherent in the target panel itself for establishing macroscopic coordinate reference. This image matrix is ​​transmitted from the camera sensor to the memory of the edge domain controller via high-speed serial interfaces such as MIPI-CSI, providing a data foundation for subsequent dual-track decoupled calculations.

[0035] For example, assuming the robot comes to a stop in front of the theoretical stopping point, the edge domain controller sends a data acquisition command to the sensing module. An asymmetric structured light emitter is activated, projecting two parallel, vertically spaced infrared laser lines with a physical spacing of 25.0 mm onto a gray metal fire alarm control panel. Simultaneously, the panel displays a 150 mm × 100 mm LCD screen. A near-infrared camera mounted at the end of the robotic arm (with a sensor pixel size of 0.005 mm / pixel, a lens physical focal length of 8.0 mm, an image resolution of 1920 × 1080, and an optical center located at pixel coordinates (960, 540)) performs the exposure.

[0036] Due to errors in physical docking, the camera's optical axis was not precisely aligned with the center of the panel, and the distance was not exactly 200 millimeters. Ultimately, the camera captured and output a single 1920×1080 pixel 8-bit grayscale image. In this image, two core feature sets can be clearly observed: one is two bright and nearly perpendicular thin lines, which are the projections of the structured light features onto the image sensor; the other is a nearly rectangular dark area with very high edge contrast, which represents the macroscopic contour features of the LCD screen.

[0037] This frame of image data, as a composite signal carrier containing dual information, is loaded into the random access memory of the digital signal processor, awaiting further processing in the next step of orthogonal feature separation and physical inverse mapping. This acquisition process utilizes the ability of a single sensor to simultaneously capture two visual cues that are completely different in nature but complementary in function under specific physical conditions (specific wavelength illumination).

[0038] S200: Based on the structured light features, determine the position information of the operation panel in a depth direction.

[0039] This step is used to extract structured light features from the composite image acquired by S100 and, using their geometric deformation under perspective projection, uniquely and definitively calculate the normal distance between the camera (i.e., the robot's end effector) and the control panel, i.e., the depth information. This process is designed to be completely immune to any translational errors in the planar direction, thereby achieving independent, high-precision calibration of the depth dimension.

[0040] Specifically, the digital signal processor in the edge domain controller first preprocesses the original image to be identified. The first step of preprocessing is feature separation. The processor executes an image segmentation algorithm, such as based on connected component analysis or morphological operations, to distinguish between elongated, high-brightness linear features (structured light) and large, blocky dark region features (macro contours) in the image. For the separated set of structured light feature pixels, in order to improve the accuracy of subsequent measurements, the processor executes a skeletonization or thinning algorithm (e.g., the Zhang-Suen thinning algorithm) to compress the light band, which was originally several pixels wide, into a precise center line with a single pixel width.

[0041] Subsequently, based on a key geometric constraint—that in the camera coordinate system, on a plane perpendicular to the optical axis—the perspective projections of two parallel line segments must intersect at infinity, meaning they remain parallel on the image plane—this is problematic. In practical applications, due to minor installation errors or panel tilt, the projected line segments may not be perfectly parallel. To eliminate this uncertainty, this method introduces a preset scanning axis.

[0042] In a preferred embodiment, the preset scan axis is defined as the horizontal scan line containing the optical center of the image. This is an absolute internal reference frame determined after camera calibration. The processor calculates the abscissa (u-coordinate) of two vertical structured light centerlines on this specific horizontal scan line (e.g., for a 1080p image, the row of pixels with v=540). By performing a straight-line fit on these two centerlines (which may be diagonal lines) (e.g., using the Random Sample Consensus (RANSAC) algorithm to enhance robustness to noise), the u-coordinate of their intersection with the horizontal scan line can be solved, denoted as […]. and The processor then calculates the absolute value of the difference between the two horizontal coordinates to obtain the pixel spacing. .

[0043] Based on the perspective projection principle (similar triangles) of the pinhole camera model, this pixel spacing measured on a specific scanning axis... Physical depth from camera to target plane There exists a strict inverse relationship between them. This relationship is determined solely by the camera's intrinsic parameters (physical focal length). Pixel size ) and the extrinsic parameters of the structured light emitter (physical spacing between the two beams) The only certainty.

[0044] Specifically, this relationship can be expressed by the formula This indicates that the processor will store the pre-stored camera model parameters ( ) and real-time measured pixel spacing Substituting this formula, the physical distance from the camera's optical center to the surface of the control panel at the current moment can be calculated, i.e., the position information in the depth direction. The robustness of this calculation process lies in the fact that, regardless of how the robot translates left and right (X-axis error) or floats up and down (Y-axis error) on the track, as long as the target remains within the camera's field of view, the pixel spacing theoretically measured on the horizontal scan line of the optical center will remain the same. Only affected by depth This eliminates the influence of [the previous sentence], thus achieving absolute decoupling of depth measurement.

[0045] For example, continuing with the example of S100 above, the digital signal processor extracts two high-brightness structured rays from a 1920×1080 image. After Sauvola local adaptive thresholding and Zhang-Suen thinning, two straight line skeletons with a width of one pixel are obtained. Subsequently, the processor calls the RANSAC algorithm to fit the pixels of these two lines to obtain their linear equations.

[0046] For example, the processor first defines the straight line model to be fitted as a standard two-dimensional plane straight line equation. The algorithm's iterative process begins by randomly selecting a minimum subset of two pixels from the skeletonized structured light pixel set to uniquely determine an initial candidate line. Subsequently, the algorithm enters the inlier discrimination phase. For each pixel in the pixel set, its orthogonal distance to the current candidate line (i.e., the perpendicular distance from the point to the line) is calculated. An exemplary interior point determination rule is: if the distance... Less than a preset distance threshold If the distance threshold is not met, the point is classified as an interior point of the candidate line. The setting is intended to tolerate minor errors introduced by image sensor noise and skeletonization algorithms; in a preferred embodiment, its value is set to 2.0 pixel units.

[0047] After identifying all points, the total number of interior points of the current candidate line is counted. This iterative process (random sampling - model fitting - interior point counting) is repeated until a preset termination condition is met. The termination condition is defined as one of the following two: First, the number of interior points of the currently found best candidate line (i.e., the line with the most interior points) exceeds a preset percentage threshold of the total number of pixels. For example, the threshold ratio can be set to 90%, indicating that a model can explain the vast majority of data points; secondly, the number of iterations has reached a preset maximum number of iterations. For example, 500 iterations are performed to prevent the algorithm from getting stuck in an infinite loop in cases of extremely poor data quality. After the iteration terminates, the algorithm selects the candidate line with the most interior points throughout the entire iteration history and refits it using all its interior points through standard least squares to obtain a more accurate line equation. This final line equation is considered the final fitting result for the structured light features and is used for subsequent pixel pitch calculations.

[0048] After obtaining the final fitting equations for the two vertical structured lights (expressed in the image coordinate system uv as follows), and After that, the processor proceeds to the final pixel pitch calculation stage. To obtain precise pitch on the preset scan axis, the processor calculates the vertical coordinate of this scan axis, which is the vertical coordinate of the camera's optical center. Substitute these values ​​into the equations of the two known lines. Through simple algebraic solutions, the x-coordinates of the intersection points of the two lines and the horizontal scan line can be determined. and The specific calculation process is as follows: as well as Finally, the processor calculates the absolute value of the difference between the two x-coordinates, i.e. This yields the unique pixel spacing, which serves as the core basis for depth calculation. .

[0049] For example, given that the optical center of the camera is located at (960, 540), the preset scan axis is... The horizontal line. The processor will Substituting the equations of the two fitted lines into the equations, we can solve for their x-coordinates on the scan line. Pixels and Pixels. Next, the critical pixel spacing is calculated. Pixels. At this point, the processor reads the pre-stored camera model parameters from its non-volatile memory: physical focal length. Millimeter, pixel physical size millimeters per pixel, and physical spacing of structured light Millimeters. Finally, substitute these values ​​into the depth calculation formula: Perform the calculation to obtain... Millimeters. This value. Millimeters represent the precise normal distance between the robot's end-effector camera and the fire alarm control panel. This high-precision, unaffected by planar translation errors, reference quantity will be used in the next step of planar position calculation.

[0050] like Figure 4 The simulation diagram of the depth calculation function shown illustrates the nonlinear inverse relationship curve between the vertical pixel spacing Δd of the structured light and the normal physical depth Z_real. The calibration point (198.0 pixels, 202.0 mm) of this embodiment is clearly marked in the figure.

[0051] like Figure 5 The simulation diagram showing the depth resolution limit further simulates the physical depth drift caused by single-pixel quantization error (i.e., extraction deviation of 1 pixel) when the image sensor extracts features.

[0052] Depend on Figure 5 It can be seen that, near the preset docking reference plane (200 mm), the extraction error of a single pixel only causes a normal physical depth deviation of about 1.0 mm. This tiny deviation is completely constrained within the preset 10 mm physical compensation tolerance limit of the end flexible push rod.

[0053] S300: Based on the macroscopic contour features, determine the position information of the operation panel in a planar direction.

[0054] This step establishes a physical coordinate system origin independent of the robot's own state to mitigate planar positioning drift caused by a lack of reference points on flat, textureless surfaces. This step utilizes the inherent macroscopic contour features of the target panel captured in S100, combined with the depth information calculated in S200, to back-project and calculate the robot's physical translational deviation in the XY plane. The specific process is as follows:

[0055] The digital signal processor again retrieves the original or pre-processed image acquired by S100. This time, its processing object is the macroscopic contour features that were previously separated, namely the outline of the liquid crystal display screen that appears as a high-contrast edge in the infrared image.

[0056] The processor executes edge detection algorithms (e.g., the Canny operator) and contour extraction algorithms (e.g., the contour discovery algorithm by Suzuki et al.) to obtain the set of all edge pixels constituting the outer frame of the liquid crystal display screen. Since the screen is typically rectangular, the processor can further perform geometric fitting on the extracted contours, for example, using Hough transform to detect lines or using a minimum area rectangle fitting algorithm, to accurately locate the pixel coordinates of the four vertices of the screen in the image coordinate system. According to a preset rule, this method selects one of the vertices as the reference physical origin. To ensure the stability and consistency of the measurement, in a preferred embodiment, this reference physical origin is defined as the upper left corner vertex of the liquid crystal display screen contour. Through geometric calculation, the processor obtains the precise pixel coordinates of this vertex in the image, denoted as... .

[0057] In some embodiments, when performing Canny edge detection, the setting of its dual thresholds (low threshold and high threshold) does not rely on fixed empirical values, but rather adopts an adaptive determination method.

[0058] For example, the processor first calculates the grayscale histogram of the entire image to be recognized, and then automatically calculates an optimal global threshold using Otsu's method. This threshold maximizes the inter-class variance. Subsequently, the high threshold of the Canny operator is set to... The low threshold is set to This approach ensures that edge detection is well-adapted to panel images under different lighting conditions.

[0059] After extracting all closed contours using the Suzuki algorithm, the processor executes a contour filtering logic to accurately identify the contours of the LCD screen from multiple potentially interfering contours. This filtering logic includes:

[0060] First, calculate the area and circumscribed rectangle of each closed contour. Second, remove all small, noisy contours with an area smaller than a preset area threshold (e.g., 5% of the total image area). Finally, among the remaining contours, calculate their aspect ratios and select the contour with the largest area and aspect ratio that is closest to the common display screen ratio (e.g., a preset target range of [1.2, 2.0]) as the final LCD screen contour.

[0061] After obtaining the coordinates of the four vertices of the contour using the minimum area rectangle fitting algorithm, to unambiguously determine which vertex is the top-left vertex, the processor establishes a temporary two-dimensional coordinate system with the top-left corner of the image (pixel coordinates (0,0)) as the origin. Then, iterates through the coordinates of these four vertices. And calculate the sum of the coordinate components of each vertex. Based on geometric relationships, it has the minimum coordinates and The vertex that is closest to the origin of the image must be the vertex that is uniquely identified as the reference physical origin, namely the top left vertex.

[0062] In the above example, the precise physical depth from the camera to the panel has been obtained from the S200. According to the inverse projection geometry of the pinhole camera model, for any pixel on the image plane, its corresponding 3D spatial point must lie on the ray connecting the camera's optical center and that pixel. When the depth plane containing this spatial point is known, its physical coordinates on the XY plane can be uniquely determined.

[0063] Specifically, the physical translation of the panel origin relative to the camera's optical axis (i.e., positional information in the planar direction). and It can be calculated using the following formula: as well as .in, These are the pixel coordinates of the camera's optical center, and are part of the camera's intrinsic parameters. The processor will extract the origin pixel coordinates in real time. Depth calculated by S200 and pre-stored camera internal parameters Substituting all the values ​​into the above formula, we can calculate the physical offset (in millimeters) of the upper left corner of the LCD screen relative to the optical center of the current camera along the X and Y axes. Together, they constitute the precise positional information of the control panel in the planar direction. This process orthogonally decomposes the tasks of depth measurement (relying on structured light) and planar localization (relying on macroscopic contours). The macroscopic contours provide orientation information of where the target is, while the structured light provides scale information of how far away the target is. The combination of the two ultimately achieves the reconstruction of the complete 3D pose of the panel in the camera coordinate system.

[0064] For example, continuing with the previous scene data, the digital signal processor processes the same frame of image, this time focusing on the dark rectangular area. Using Canny edge detection and contour fitting, the processor identifies the outline of the LCD screen and calculates the pixel coordinates of its four vertices. Based on preset rules, it selects the top-left vertex as the reference physical origin and determines its coordinates as follows: Pixels. At this point, the processor calls the calculation result of S200, i.e., the precise depth. Millimeters. Simultaneously, camera intrinsics are read from memory: physical focal length. millimeters, pixel size millimeters per pixel, and optical center coordinates Next, the processor performs the inverse mapping calculation of the planar position. For the offset in the X-axis direction, the calculation process is as follows: This simplifies to Millimeters. The calculation process for the offset in the Y-axis direction is as follows: This simplifies to Millimeters. Therefore, the processor determines the planar position information of the operation panel as follows: its physical origin (the upper left corner of the LCD screen) is located in the current camera coordinate system. The position is measured in millimeters. These two values ​​precisely quantify the translational error in the planar direction caused by macroscopic inaccuracy in robot docking.

[0065] S400: Based on the position information in the depth direction and the position information in the planar direction, determine the final three-dimensional spatial coordinates of a target operation area on the operation panel.

[0066] This step fuses and transforms the dispersed depth and planar position information calculated in the previous steps, ultimately generating a 3D coordinate system of the target point in the robot arm's global coordinate system that can be directly executed by the robot's motion controller. This process involves cascaded transformations of multiple coordinate systems. Specifically, this step can be broken down into three sub-steps.

[0067] First, target point synthesis in the camera coordinate system. The edge domain controller first integrates the calculation results of S200 and S300. S200 provides depth information. S300 provides planar position information. These two elements together define the three-dimensional coordinates of the panel's physical origin in the camera coordinate system, which can be represented as a homogeneous coordinate vector. The controller then reads the target area information, such as the mute button, from its task cache. This information comes from a pre-set database that stores the precise physical offset vector of all buttons relative to the panel's physical origin (i.e., the upper left corner of the LCD screen). For example, the database specifies that the center of the mute button is located 50 mm to the right and 20 mm below the origin, meaning its physical offset vector is... Millimeters. The controller uses simple vector addition to combine the origin coordinates with the button offset vector to calculate the three-dimensional coordinates of the target button center in the current camera coordinate system. (Note the consistency of coordinate system orientation).

[0068] Second, coordinate system transformation. Understandably, all coordinates so far are relative to the camera's local coordinate system. To enable the robotic arm to move to that point, they must be transformed to the robotic arm's base coordinate system (global coordinate system). The edge domain controller reads the current joint angles from the encoders of each joint of the six-axis robotic arm in real time and calculates the homogeneous transformation matrix of the current camera coordinate system relative to the base coordinate system using the Forward Kinematics (FK) algorithm. .this The matrix contains the camera's rotational attitude and translational position relative to the base. The controller performs one matrix multiplication operation: The result of the calculation It is The column vector, whose first three terms are the absolute three-dimensional spatial coordinates of the target button center in the coordinate system of the robotic arm base, is the final three-dimensional spatial coordinate to be determined by this method.

[0069] Third, execute the command generation. Obtain global coordinates. The edge domain controller then uses this as input to pass it to the inverse kinematics (IK) solver of the robotic arm. In a preferred embodiment, the solver employs an iterative numerical solution based on the pseudo-inverse of the Jacobian matrix.

[0070] Specifically, in each control cycle, the solver first calculates the current pose of the robotic arm's end effector and the target pose. The error vector between the two positions is calculated. Then, the Jacobian matrix of the robotic arm in that pose is calculated, and its pseudo-inverse is computed using singular value decomposition (SVD) or damped least squares. By left-multiplying the error vector by the pseudo-inverse of the Jacobian matrix, an incremental velocity command in joint space is obtained. This command is integrated and used to update the angles of each joint, thereby driving the robotic arm to approach the target pose.

[0071] This iterative process is repeated within each control cycle until the error between the end-effector pose and the target pose is less than a preset tolerance threshold (e.g., position error less than 0.1 mm). Furthermore, the solver incorporates singularity avoidance logic. During the iterative calculation of the pseudo-inverse of the Jacobian matrix, the system monitors the condition number of the matrix. When the condition number exceeds a preset threshold, indicating that the robotic arm is approaching a singular posture, the system automatically increases the damping factor or constrains the motion velocity in the task space to ensure the numerical stability of the solution, prevent runaway joint velocities, and thus guarantee the practical feasibility of the entire closed-loop control process.

[0072] The inverse kinematics solver will calculate one or more sets of kinematics solutions that will enable the robotic arm's end effector to reach the target point. The controller selects the optimal solution and generates corresponding servo motor drive commands, sending them to the drivers of each joint. This drives the robotic arm to precisely move above the target button, ready to execute the pressing operation. The entire process thus forms a complete perception-decision-execution closed loop.

[0073] For example, the edge domain controller integrates the results of the preceding steps. It knows the coordinates of the panel origin in the camera frame. Assuming the received command is to operate the mute button, the controller queries its internal database to determine the physical offset of the button relative to the top-left corner of the LCD screen. millimeters Millimeters. The controller performs the first step of vector synthesis, calculating the coordinates of the target button in the camera coordinate system: Meanwhile, the controller reads the real-time angles of the six joints of the robotic arm, runs a forward kinematics model, and calculates the current transformation matrix from the camera to the base. Assume the calculated matrix represents the camera in the base coordinate system. At a distance of millimeters, and with the optical axis parallel to the X-axis of the base. The controller performs matrix multiplication. This will result in something similar to The result (the specific value depends on the rotation matrix). This vector It refers to the absolute coordinates of the center of the mute key in the robot world.

[0074] Finally, this coordinate is fed into the inverse kinematics solver, which outputs the precise angles that the six joint motors need to rotate, for example... These angle commands are sent to the servo drive, and the robotic arm moves precisely accordingly. The physical push rod at its end is finally aligned with the center of the mute key, completing the entire precise identification and positioning process.

[0075] To more clearly illustrate the data flow relationships throughout the entire process described above, please refer to [link / reference]. Figure 3 . Figure 3 This is a dual-track decoupled physical inverse mapping data flow diagram provided in an embodiment of this application. For example... Figure 3 As shown, the entire recognition process achieves dual-track orthogonal decoupling of feature extraction and localization calculation:

[0076] The first track extracts the pixel spacing Δd using structured light features, performs depth dimension calibration, and calculates Z_real independently. The second track extracts the absolute physical origin image plane coordinates (u_org, v_org) using macroscopic contour features, and forces the Z_real calculated from the first track as the inverse projection scaling factor, thereby completing the planar dimension calibration and calculating the planar translations X_offset and Y_offset. The data from the two independent tracks are finally fused in the target coordinate synthesis stage, and combined with the feedback of the forward kinematics matrix, an absolute servo 3D coordinate is generated. This data topology flow mechanism eliminates the perspective confusion problem that is easily caused by a single visual feature.

[0077] In one embodiment, this application also provides a fire alarm control panel accurate identification system for performing the methods described in any of the foregoing embodiments. This system can be a physical entity, such as a hardware and software assembly integrated within a sliding rail robot.

[0078] Please see Figure 2 , Figure 2 This is a hardware and software co-architecture diagram of a fire alarm control panel accurate identification system provided in an embodiment of this application. Figure 2 As shown, the system is clearly divided at the physical level into a sensing / input end, an edge domain controller (DCU) as the computing / decision end, and an execution / output end. At the logical functional level, the system includes an image acquisition module, a depth position determination module, a planar position determination module, and a target coordinate synthesis module. The following section combines... Figure 2 These modules and their hardware-software mapping relationships are explained.

[0079] The system includes an image acquisition module, a depth position determination module, a planar position determination module, and a target coordinate synthesis module.

[0080] The image acquisition module, physically mounted on the aforementioned sensing module fixed to the end effector of the robotic arm (including a near-infrared camera and an asymmetric structured light emitter), along with its connected drive and interface circuits, is configured to, upon receiving a trigger signal from the upper-level control system, control the asymmetric structured light emitter to actively project structured light features with preset geometric constraints onto the target fire alarm control panel, while simultaneously controlling the near-infrared camera for exposure and image data acquisition. The module outputs a digitized image to be identified, designed to simultaneously include structured light features for depth measurement and macroscopic contour features for planar localization, and transmits this image to the subsequent processing module via a data bus (such as MIPI-CSI or Ethernet).

[0081] The depth positioning module, physically housed within a digital signal processor (DSP) in an edge domain controller and its running firmware algorithm, takes as input the image to be identified generated by the image acquisition module. Internally, this module integrates a series of image processing and geometric calculation algorithms specifically configured to handle structured light features within the image. It first extracts the precise centerline of the structured light using image segmentation and skeletonization algorithms. Then, on a pre-defined scanning axis, such as the horizontal axis of the optical center, it measures the pixel spacing between structured light features (e.g., two parallel lines). Finally, the module calls a pre-stored depth calculation formula based on a pinhole camera model, using the measured pixel spacing and pre-defined camera / structured light physical parameters to calculate the normal distance between the camera and the control panel, i.e., the positional information in the depth direction. The module's output is a high-precision floating-point value representing the physical depth.

[0082] The planar position determination module, physically housed within a digital signal processor (DSP) or a main control ARM chip in an edge domain controller, runs a different set of algorithms than the depth position determination module. Its input is the image to be identified, along with the precise depth value output by the depth position determination module. This module is specifically configured to process macroscopic contour features in the image. Through edge detection, contour extraction, and geometric fitting algorithms, it locates the contour of a pre-defined physical component (such as an LCD screen) in the image and extracts the pixel coordinates of a feature point (such as the top-left corner vertex) as a reference physical origin. Then, using a camera inverse projection model, the module combines these pixel coordinates, the input depth value, and pre-defined camera intrinsic parameters to calculate the physical translation of this physical origin relative to the camera's optical axis along the X and Y axes—that is, its positional information in the planar direction. The module's output is a two-dimensional vector representing the physical translation error.

[0083] The target coordinate synthesis module, physically carried by the edge domain controller's main ARM chip and its running application software, serves as the decision-making and execution interface for the entire system. Its inputs include the depth value output from the depth position determination module, the planar position vector output from the planar position determination module, a database containing the offset of the target operation area (e.g., a button) relative to the physical origin, and real-time pose information from the robotic arm controller. This module is configured to perform a series of coordinate transformations and synthesis operations. First, it fuses the depth and planar position information into the three-dimensional coordinates of the panel's physical origin in the camera coordinate system. Then, it adds the relative offset of the target button to obtain the target button's coordinates in the camera coordinate system. Finally, using the robotic arm's real-time forward kinematics matrix, it transforms these coordinates from the camera coordinate system to the robotic arm's global base coordinate system, thereby generating the final three-dimensional spatial coordinates that the robot can execute. The module's output is this final three-dimensional coordinate system, which is then passed to the robot's motion planning and control system to complete the physical operations.

[0084] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0085] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the process. Figure 1 One or more processes and / or frames Figure 1 A device that specifies the function in one or more boxes.

[0086] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an implementation flow. Figure 1 One or more processes and / or frames Figure 1 Manufactured goods of instruction devices that specify functions in one or more boxes.

[0087] These computer program instructions can also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or frames Figure 1 The steps for the function specified in one or more boxes.

[0088] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0089] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit described above can be implemented in hardware.

Claims

1. A method for accurate identification of the operation panel of a fire alarm control panel, characterized in that, include: Acquire an image to be identified that includes a fire alarm control panel, wherein the image to be identified includes a structured light feature formed on the control panel and projected by an asymmetric structured light emitter, and a macroscopic contour feature of the control panel. Based on the structured light features, the position information of the operation panel in a depth direction is determined; Based on the aforementioned macroscopic contour features, the position information of the operation panel in a planar direction is determined; Based on the position information in the depth direction and the position information in the planar direction, the final three-dimensional spatial coordinates of a target operation area on the operation panel are determined.

2. The method according to claim 1, characterized in that, The structured light feature is a parallel light feature consisting of at least two parallel light bands in physical space; The step of determining the position information of the operation panel in a depth direction based on the structured light features includes: On a preset scanning axis in the image to be identified, the pixel spacing between the at least two light bands is extracted; Based on the pixel spacing and a preset camera model parameter, the position information in the depth direction is determined.

3. The method according to claim 2, characterized in that, The preset scanning axis is the horizontal scanning line where the optical center of the image to be identified is located.

4. The method according to claim 1, characterized in that, The macroscopic contour feature is the contour of a preset physical component on the operation panel; The step of determining the position information of the operation panel in a planar direction based on the macroscopic contour features includes: The outline of the preset physical component is identified in the image to be identified, and the pixel coordinates of a reference physical origin on the outline are extracted. The position information in the planar direction is determined based on the pixel coordinates of the reference physical origin, the position information in the depth direction, and a preset camera model parameter.

5. The method according to claim 4, characterized in that, The preset physical component is a liquid crystal display screen, and the reference physical origin is a preset vertex of the outline of the liquid crystal display screen.

6. The method according to claim 4, characterized in that, The step of determining the final three-dimensional spatial coordinates of a target operation area on the operation panel based on the position information in the depth direction and the position information in the planar direction includes: The position information in the planar direction is combined with the position information in the depth direction to obtain the three-dimensional spatial coordinates of the reference physical origin in the camera coordinate system; Obtain a preset physical offset vector of the target operating region relative to the reference physical origin; The three-dimensional spatial coordinates of the reference physical origin in the camera coordinate system are combined with the preset physical offset vector to obtain the three-dimensional spatial coordinates of the target operating area in the camera coordinate system. The three-dimensional spatial coordinates of the target operating area in the camera coordinate system are transformed to a robotic arm base coordinate system to obtain the final three-dimensional spatial coordinates.

7. A precise identification system for the operation panel of a fire alarm control panel, characterized in that, include: An image acquisition module is used to acquire an image to be identified, which includes an operation panel of a fire alarm host. The image to be identified includes a structured light feature formed on the operation panel and projected by an asymmetric structured light emitter, as well as a macroscopic contour feature of the operation panel. A depth position determination module is used to determine the position information of the operation panel in a depth direction based on the structured light features; A planar position determination module is used to determine the position information of the operation panel in a planar direction based on the macroscopic contour features; A target coordinate synthesis module is used to determine the final three-dimensional spatial coordinates of a target operation area on the operation panel based on the position information in the depth direction and the position information in the planar direction.

8. The system according to claim 7, characterized in that, The structured light feature is a parallel light feature consisting of at least two parallel light bands in physical space; The depth position determination module is used for: On a preset scanning axis in the image to be identified, the pixel spacing between the at least two light bands is extracted; Based on the pixel spacing and a preset camera model parameter, the position information in the depth direction is determined.

9. The system according to claim 7, characterized in that, The macroscopic contour feature is the contour of a preset physical component on the operation panel; The planar position determination module is used for: The outline of the preset physical component is identified in the image to be identified, and the pixel coordinates of a reference physical origin on the outline are extracted. The position information in the planar direction is determined based on the pixel coordinates of the reference physical origin, the position information in the depth direction, and a preset camera model parameter.

10. The system according to claim 9, characterized in that, The target coordinate synthesis module is used for: The position information in the planar direction is combined with the position information in the depth direction to obtain the three-dimensional spatial coordinates of the reference physical origin in the camera coordinate system; Obtain a preset physical offset vector of the target operating region relative to the reference physical origin; The three-dimensional spatial coordinates of the reference physical origin in the camera coordinate system are combined with the preset physical offset vector to obtain the three-dimensional spatial coordinates of the target operating area in the camera coordinate system. The three-dimensional spatial coordinates of the target operating area in the camera coordinate system are transformed to a robotic arm base coordinate system to obtain the final three-dimensional spatial coordinates.

Images