An instrument display parameter recognition method based on target detection and character recognition fusion

Abstract

The application discloses an instrument display parameter recognition method based on target detection and character recognition fusion. The method comprises the following steps: pre-processing collected instrument display interface images to obtain to-be-recognized image frames; transforming a plurality of boundary points into the to-be-recognized image frames through a homography mapping matrix, determining a graphic parameter detection area and a text parameter recognition area, and thus determining a graphic parameter area detection result and a text parameter result; field-level fusing the text parameter result and the graphic parameter to obtain a current frame recognition value; storing the current frame recognition value in a historical observation window in a shooting time sequence; filtering a continuous frame recognition value sequence to output a parameter recognition result; and the application solves at least one of the following technical problems in the prior art: recognition area misplacement, difficulty in stably reading graphic parameters through character recognition, interference of graphic background on whole-image character recognition, and easy jumping of single-frame recognition results.

Description

Technical Field

[0001] This invention relates to the fields of display information recognition and computer vision technology, and more specifically, to a method for recognizing instrument display parameters based on the fusion of target detection and character recognition. Background Technology

[0002] In the fields of industrial production operation and maintenance, equipment condition monitoring and automated testing, instruments are the intuitive carriers of core data, and the accurate acquisition of their display parameters is crucial. Traditional manual recording methods are not only inefficient, but also easily affected by the observation angle, personnel attention and subjective readings, making it difficult to meet the monitoring requirements of real-time and continuous monitoring.

[0003] While existing non-contact instrument recognition solutions based on external image acquisition are gradually becoming more widespread, they are still limited by two major bottlenecks at the perception level: First, the recognition area adaptability is poor. Affected by installation angle deviation, equipment vibration drift, and lens perspective distortion, the preset fixed Region of Interest (ROI) is often misaligned with the actual display area, directly leading to the failure of subsequent graphic element detection and text cropping. Second, the stability of graphic parameter reading is insufficient. Graphical parameters such as pointers, liquid levels, and bubbles rely on the geometric relationship between the reference mark, scale mark, and graphic key points. Conventional optical character recognition (OCR) is easily affected by complex backgrounds and scale lines. The accuracy of traditional geometric calculation drops sharply in scenarios with rotation, occlusion, and uneven lighting.

[0004] Meanwhile, the processing logic of the existing solutions also has obvious shortcomings: on the one hand, the text recognition efficiency is low, the computational load of whole-image text detection is large and the real-time performance is poor, and it is impossible to use the prior information of fixed instrument layout to filter out graphic noise; on the other hand, the temporal output stability is poor, and the single-frame recognition results are easily affected by reflection, blurring, and model false detection, resulting in instantaneous outliers. Existing filtering methods mostly rely on simple mean or Kalman filtering, lack anomaly identification mechanism, and it is difficult to suppress noise while responding to changes in real parameters in a timely manner.

[0005] In summary, existing technologies lack a comprehensive solution that can effectively integrate the advantages of object detection and character recognition while simultaneously addressing issues such as region misalignment, difficulty in image reading, background interference, and inter-frame transitions. Summary of the Invention

[0006] This invention provides a method for recognizing instrument display parameters based on the fusion of target detection and character recognition, in order to solve at least one of the technical problems in existing instrument visual recognition, such as misalignment of recognition areas caused by installation deviation of image acquisition device, perspective distortion of the image, difficulty in stably reading graphic parameters through character recognition, susceptibility of whole-image character recognition to interference from graphic background, and easy jump in single-frame recognition results.

[0007] According to one aspect of the present invention, a method for recognizing instrument display parameters based on the fusion of target detection and character recognition is provided. The method may include: acquiring an image of the instrument display interface; preprocessing the image of the instrument display interface to obtain an image frame to be recognized; wherein the preprocessing includes image format conversion, resolution normalization, and image denoising. Based on the correspondence between feature points between the reference image and the image frame to be identified, a homography mapping matrix is ​​calculated. Multiple boundary points of the preset recognition region are transformed into the image frame to be identified through the homography mapping matrix. Based on the transformed boundary points, the dynamically corrected graphic parameter detection region and text parameter recognition region are determined. Among them, feature points include corner points, boundary points, stable texture points or preset marker points. The graphic parameter detection region is input into the target detection model to detect the baseline marker, graphic key points, and scale markers, obtaining the graphic parameter region detection result. Based on the graphic parameter region detection result, the graphic parameters are calculated. Based on the text parameter recognition region, the text region is cropped from the image frame to be recognized and input into the character recognition model deployed using inference optimization in a batch manner to obtain the character recognition result. The character recognition result is post-processed to obtain the text parameter result. The text parameter result and the graphic parameter are fused at the field level to obtain the recognition value of the current frame. The current frame identification value is stored in the historical observation window in chronological order of shooting time. The historical observation window is used to store a sequence of continuous frame identification values, including the identification values ​​of the current frame and several historical frames. Post-processing includes legal character filtering, symbol resolution, numerical formatting, unit conversion, and preset format conversion. The continuous frame identification value sequence is filtered, and the parameter identification result is output.

[0008] Optionally, the graphic parameters include a first graphic parameter and a second graphic parameter. Based on the graphic parameter region detection results, the graphic parameters are calculated, including: filtering scale markers located on both sides of the graphic parameter detection region in the graphic parameter region detection results that have the same numerical attributes; calculating the coordinates of the center point of the scale markers to generate scale reference points; extracting reference markers from the graphic parameter region detection results and calculating the positional offset of the reference markers relative to adjacent scale reference point data; obtaining the first graphic parameter through a linear interpolation algorithm based on the proportional relationship between the positional offset and the preset scale spacing data; extracting the coordinates of graphic key points or scale reference points from the graphic parameter region detection results, constructing direction vector data, calculating the angle between the direction vector data and the preset reference direction data, and assigning a sign attribute based on the quadrant in which the vector is located to obtain the normalized second graphic parameter; and integrating the first graphic parameter and the second graphic parameter into a single graphic parameter.

[0009] Optionally, the filtering of legal characters includes: establishing a set of legal characters for the preset formats of different instrument display parameters; for numerical parameters, retaining numeric characters, decimal points, minus signs and preset separators, and removing other interfering characters; for composite format text parameters using degrees, minutes and seconds, converting them into floating-point data format of decimal degrees according to preset conversion rules.

[0010] Optionally, the continuous frame identification value sequence is filtered to output parameter identification results, including: if the deviation between the current frame identification value and the predicted value does not exceed a preset deviation threshold, the current frame identification value is determined as the output result, and the historical observation window is updated; if the deviation exceeds the preset deviation threshold and the number of consecutive abnormal frames does not reach a preset number, the previous valid output value is kept unchanged, and the number of consecutive abnormal frames is updated; wherein, the number of consecutive abnormal frames is the cumulative value of the number of frames in the continuous frame identification value sequence where the deviation exceeds the standard; if the deviation exceeds the preset deviation threshold and the number of consecutive abnormal frames reaches a preset number, the current frame identification value is determined as the output result, and the historical observation window and the number of consecutive abnormal frames are updated using the current frame identification value.

[0011] According to one aspect of the present invention, an instrument display parameter recognition system based on the fusion of target detection and character recognition is provided. The system includes: an image acquisition module for acquiring images of the instrument display interface, and preprocessing the instrument display interface images to obtain image frames to be recognized; wherein the preprocessing includes image format conversion, resolution normalization, and image denoising; The region correction module is used to calculate the homography mapping matrix based on the correspondence between feature points between the reference image and the image frame to be identified. It transforms multiple boundary points of the preset recognition region into the image frame to be identified through the homography mapping matrix, and determines the dynamically corrected graphic parameter detection region and text parameter recognition region based on the transformed boundary points. The feature points include corner points, boundary points, stable texture points or preset marker points. The graphic parameter recognition module is used to input the graphic parameter detection area into the target detection model, detect the reference mark, graphic key points and scale mark, and obtain the graphic parameter area detection result; The graphic parameter calculation module is used to calculate graphic parameters based on the graphic parameter region detection results; The character recognition module is used to identify regions based on text parameters. It crops out text regions from the image frame to be recognized and inputs the text regions into the character recognition model deployed by inference optimization in a batch input manner to obtain character recognition results. The data processing and output module is used to post-process the character recognition results to obtain text parameter results; to perform field-level fusion of the text parameter results and graphic parameters to obtain the current frame recognition value; to store the current frame recognition value in the historical observation window in the order of shooting time; wherein, the historical observation window is used to store a continuous frame recognition value sequence consisting of the recognition values ​​of the current frame and several historical frames; post-processing includes legal character filtering, symbol parsing, numerical formatting, unit conversion and preset format conversion; to perform filtering processing on the continuous frame recognition value sequence, and to determine and output the current frame output result based on the continuous frame recognition value sequence.

[0012] Optionally, the region correction module is also used to: establish a correspondence between feature points between the reference image and the image frame to be identified based on corner points, boundary points, stable texture points, or preset marker points.

[0013] The beneficial effects of this invention are: This invention proposes a method for recognizing instrument display parameters based on the fusion of target detection and character recognition. By dynamically correcting the graphic and text detection areas through homography mapping, it effectively eliminates recognition misalignment caused by installation deviations and perspective distortion, significantly improving robustness in complex environments. It utilizes a target detection model to simultaneously extract reference markers, graphic key points, and scale markings, combined with a dual graphic parameter calculation mechanism, overcoming the limitations of traditional OCR in graphic parameter recognition and greatly improving reading accuracy. Through batch OCR after cropping and a field-level fusion strategy, it greatly reduces computational overhead while avoiding graphic noise interference, meeting the real-time processing requirements of multiple parameters. In particular, the introduced three-level state machine filtering mechanism based on historical observation windows can accurately identify instantaneous outliers, suppressing output jitter while responding promptly to changes in real parameters, achieving high stability and reliability of continuous frame parameter output. Attached Figure Description

[0014] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings: Figure 1 This is a flowchart of an instrument display parameter recognition method based on the fusion of target detection and character recognition according to an embodiment of the present invention; Figure 2 This is a schematic diagram of an instrument display interface before identification area correction according to an embodiment of the present invention; Figure 3 This is a schematic diagram of an instrument display interface after identification area correction according to an embodiment of the present invention; Figure 4 This is a schematic diagram of a structured parameter output according to an embodiment of the present invention. Detailed Implementation

[0015] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0016] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0017] Example According to an embodiment of the present invention, a method for identifying instrument display parameters based on the fusion of target detection and character recognition is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system containing at least one set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0018] Figure 1 This is a flowchart of an instrument display parameter recognition method based on the fusion of target detection and character recognition according to an embodiment of the present invention, as shown below. Figure 1 As shown, the method may include the following steps: Step S101: Acquire the image of the instrument display interface, and preprocess the image of the instrument display interface to obtain the image frame to be identified; wherein, the preprocessing includes image format conversion, resolution normalization and image denoising.

[0019] In the technical solution provided by step S101 of the present invention, a unified and high-quality image input is provided for subsequent dynamic correction of recognition areas, detection of graphic parameters and character recognition, eliminating the differences in format, scale, noise and other aspects of the original acquired images, and ensuring that the entire recognition pipeline is processed under a unified standard.

[0020] Specifically, the original image of the instrument display interface is acquired through an external image acquisition device. The instrument display interface includes a graphical parameter display area and a text parameter display area, encompassing various display elements such as pointer indicators, scale markings, numerical parameters, and composite text parameters. Because the original acquired instrument display interface images may have inconsistent formats, varying resolutions, and noise, directly using them for subsequent recognition can affect the accuracy of feature point matching, region correction, and the effectiveness of target detection and character recognition. Therefore, this step performs preprocessing operations on the original acquired instrument display interface images, including format conversion, resolution normalization, noise reduction, and brightness correction.

[0021] For example, suppose the first The original image of the instrument display interface acquired in the frame is Its image size is Before the acquired instrument display interface image enters the recognition process, preprocessing operations such as format conversion, resolution normalization, noise reduction, and brightness correction are performed sequentially to obtain the image frame to be recognized. The processing procedure can be expressed as follows:

[0022] in, Indicates image format conversion. This indicates resolution normalization. Indicates image denoising. Indicates brightness correction; preprocessed image frame to be recognized. It serves as a unified input for region correction, graphic parameter detection, and character recognition.

[0023] Specifically, image format conversion refers to converting the acquired instrument display interface images into a standardized format agreed upon in subsequent processing procedures, ensuring that the instrument display interface images meet unified requirements in terms of organization and color space representation; and eliminating compatibility issues caused by differences in output formats of different acquisition devices.

[0024] Resolution normalization refers to standardizing the instrument display interface image from different acquisition resolutions to a preset input size; for example, assuming the coordinates of the instrument display interface image are... If the model input size is W×H and the instrument display interface image size is W0×H0, then the normalized coordinates (x,y) satisfy:

[0025] This operation ensures that the recognition area, detection result, and cropping area are in a unified image coordinate system under different acquisition resolutions, providing a scale consistency basis for subsequent homography mapping and region correction.

[0026] Image denoising refers to performing noise reduction processing on an image to suppress random noise, sensor noise, and environmental interference noise introduced during the acquisition process, thereby improving image quality and providing cleaner image data for subsequent feature point extraction, target detection, and character recognition. In a preferred embodiment, this step may further include a brightness correction operation to eliminate the impact of uneven illumination on recognition accuracy.

[0027] Step S102: Based on the correspondence between feature points between the reference image and the image frame to be identified, calculate the homography mapping matrix, transform multiple boundary points of the preset recognition region into the image frame to be identified through the homography mapping matrix, and determine the dynamically corrected graphic parameter detection region and text parameter recognition region based on the transformed boundary points; wherein, the feature points include corner points, boundary points, stable texture points or preset marker points.

[0028] In the technical solution provided by step S102 of the present invention, the core objective is to solve the problem of misalignment of the recognition area caused by the installation angle deviation of the image acquisition device, the position drift of the equipment due to vibration, or the perspective distortion of the lens. By performing homography dynamic correction on the preset recognition area, the subsequent graphic parameter detection area and text parameter recognition area can be adaptively aligned to the actual instrument display content of the current frame.

[0029] Specifically, the core of this step lies in performing dynamic correction on the preset recognition area. Dynamic correction refers to using the preset recognition area in the reference image as a benchmark, extracting the stable feature point correspondence between the reference image and the current image frame to be recognized, calculating the homography mapping matrix to compensate for the regional misalignment caused by installation angle deviation, equipment vibration drift and lens perspective distortion, and adaptively aligning the preset recognition area to the actual instrument display content of the current frame. Its update adopts a trigger mechanism, which does not require repeated calculations frame by frame, thus balancing correction accuracy and processing efficiency.

[0030] Specifically, the correspondence between feature points between the reference image and the current image frame to be identified is extracted. Feature points are iconic points that can stably characterize the spatial position and geometric structure of the instrument display area, including corner points, boundary intersections, stable texture points, or preset marker points of the instrument display interface.

[0031] For example, the i-th feature point in the reference image is represented as:

[0032] Image frame I to be identified t The corresponding feature points are represented as follows:

[0033] The homography mapping relationship between the reference image coordinate system and the coordinate system of the image frame to be recognized is expressed as follows:

[0034] in, The homogeneous coordinate scale factor. Indicates the first The homography mapping matrix corresponding to the frame; It can be represented as:

[0035] To reduce the impact of mismatched feature points on the correction results, a robust estimation method is used to remove outlier correspondences. The optimized homography mapping matrix is ​​solved by minimizing the robust loss function.

[0036] in, This indicates the number of feature points involved in the matching. Represents the robust loss function. This represents the homogeneous coordinate normalization function, used to convert homogeneous coordinates into image plane coordinates; through robust estimation, it can reduce the impact of mismatched feature points on the region mapping results.

[0037] Obtain the composition of the preset recognition region in the reference image; the preset recognition region is stored in the form of a homogeneous boundary point set, assuming that the first boundary point in the reference image coordinate system is... The preset recognition areas are It consists of several boundary points, then:

[0038] in, Indicates the first The first preset recognition area homogeneous boundary points This indicates the number of boundary points in the region; for rectangular regions, it is usually taken as... That is, it is determined by the four corner points: upper left, upper right, lower right, and lower left; the preset recognition area includes the entire graphic parameter detection area. and N text parameter recognition regions :

[0039] in, This indicates the area for detecting graphic parameters, covering the graphic indicator area, reference markers, scale markers, and the area containing key points of the graphic. Indicates the first Each text parameter recognition region Indicates the number of parameter display areas for text-based instruments.

[0040] The boundary points of the preset recognition area are mapped using the optimized homography mapping matrix. Transform to the coordinate system of the current image frame to be recognized; for any preset boundary point The homogeneous coordinates after mapping are:

[0041] After normalization, the coordinates of the non-homogeneous image are obtained:

[0042] This mapping relationship is applied to the preset recognition area. For each boundary point in the data, the dynamically corrected recognition region in frame t is obtained as follows:

[0043] in,

[0044] Based on the transformed boundary points, determine the recognition area range after dynamic correction: extract the set of abscissas of the four corner points after transformation. With the set of ordinates The rectangular clipping boundary of the current frame is calculated as follows:

[0045]

[0046] This yields the cropping box for the k-th recognition region in the current frame:

[0047] The corresponding The cropping frame is the dynamically corrected image parameter detection area, corresponding to... The cropping box is the dynamically corrected text parameter recognition area.

[0048] Triggered update decision-making for dynamic correction ensures region alignment efficiency. First, the homography mapping error of the current frame is calculated:

[0049] When mapping error Exceeding the preset error threshold Or, the offset Δt of the current frame key point position relative to the reference position / previous frame position exceeds the preset region update threshold. If so, re-execute the above homography region mapping process; ≤ And Δt≤ If the recognition area is used, the recognition area of ​​the previous frame will be reused, and there is no need to calculate it again; among them, This represents the homography mapping error threshold. This indicates the offset of the current frame's keypoint position relative to the reference position or the position in the previous frame. This indicates the region update threshold.

[0050] The dynamically corrected graphic parameter detection area will serve as the input for the subsequent target detection model, used to detect baseline markers, graphic key points, and scale markers; the dynamically corrected text parameter recognition area will be used to accurately crop text regions from the image frame to be recognized, avoiding interference from the graphic background in the whole image OCR, and providing high-quality input for subsequent batch character recognition.

[0051] Step S103: Input the graphic parameter detection area into the target detection model, detect the reference marker, graphic key points and scale markers, and obtain the graphic parameter area detection result.

[0052] In the technical solution provided in step S103 of the present invention, the three types of graphic elements—reference marks, graphic key points, and scale marks—are accurately located within the dynamically corrected graphic parameter detection area, providing reliable input of detection elements for the geometric calculation of graphic parameters in the subsequent step S104.

[0053] Specifically, extract the dynamically corrected image parameter detection region in the current frame t. (Corresponding to preset area) The mapping result), from the image frame I to be identified t The corresponding region image is cropped from the middle; let the detection region of the dynamically corrected graphic parameters in frame t be... The corresponding cropped image is:

[0054] in, This indicates that image cropping is performed based on the specified area; the cropped image As the sole input to the object detection model, it does not involve other areas of the entire image frame to be identified, thus effectively avoiding interference from irrelevant elements such as text regions, borders, and reflective points on the detection of graphic elements.

[0055] Crop the image Input is fed into an object detection model deployed using inference optimization: The object detection model performs object detection on the cropped image and outputs the detection results.

[0056]

[0057] in, This represents the object detection model. This represents the m-th detected target. Indicates the target category, Indicates the position of the detection box. Indicates the detection confidence level. Indicates the number of targets to be detected.

[0058] The target detection model takes the image of the graphic parameter region as input and uses the reference marker, graphic key points, and discrete scale markers as the detection targets, outputting the corresponding category, location, and confidence score. The reference marker represents the reference position of the graphic parameters (e.g., the pointer of a pointer instrument, the liquid level line of a level gauge, the bubble center of a bubble indicator), and is the core reference point for calculating the first and second graphic parameters. Graphic key points characterize the geometric structure of the graphic parameter region (e.g., the rotation center, pointer endpoints, endpoints of graphic elements), and are mainly used for constructing the direction vector of the second graphic parameter. The scale markers are numerical markers with fixed intervals in the graphic parameter region, used to establish the scale reference relationship required for parameter calculation, and are mainly used for linear interpolation calculation of the first graphic parameter.

[0059] Detection boxes output by the object detection model Represented as:

[0060] in, Indicates the coordinates of the center point of the detection box. and Let represent the width and height of the detection box, respectively; for the scale marker target, its category value and center point coordinates are obtained based on the detection results. Let the r-th scale marker detection box be . The coordinates of its center point are:

[0061] The coordinates of this center point will serve as the basis for matching the same scale value, constructing the scale reference point, and calculating the projection distance in the subsequent step S104.

[0062] The complete set of detection results output by the object detection model This is the result of the graphic parameter area detection obtained in this step; the detection result only includes the detection elements such as the category, position and confidence level of the benchmark, graphic key points and scale marks.

[0063] The target detection model in this invention is a detection network used to simultaneously detect three types of graphic elements—reference markers, graphic key points, and scale markers—from the dynamically corrected graphic parameter detection region image. It takes the cropped graphic parameter detection region image as its sole input and outputs the category, detection box position, and confidence level of each detected target, thus forming the graphic parameter region detection result. This model is deployed using an inference optimization approach, which includes computation graph optimization, adjacent operator fusion, convolution and normalization layer fusion, low-precision computation, storage resource reuse, and execution operator optimization. This accelerates the inference process of graphic parameter detection, enabling it to adapt to the real-time processing requirements of continuous image frames and providing a reliable element basis for subsequent calculation of the first and second graphic parameters based on the detection results.

[0064] Step S104: Calculate graphic parameters based on the region detection results.

[0065] In the technical solution provided by step S104 of the present invention, based on the graphic parameter region detection results obtained in step S103, graphic parameters are obtained through geometric calculation; wherein, the graphic parameters include first graphic parameters and second graphic parameters. Since the target detection model output in step S103 only outputs detection elements such as the category, position, and confidence level of the benchmark, graphic key points, and scale markings, rather than the final parameter values; this step S104 is the core calculation step that transforms the detection elements into actual instrument readings.

[0066] Specifically, the execution process of this step includes two parallel branches: the calculation of the first graphic parameter and the calculation of the second graphic parameter. The first graphic parameter calculation branch (position / displacement parameters) is as follows: the calculation of the first graphic parameter is based on the center point position of the left and right scale marks with the same value, the distance between adjacent scale marks, and the relative position of the reference mark, and is achieved through linear interpolation.

[0067] First, match the scale markers with the same value. The scale markers are numerical markers with fixed intervals in the graphic parameter area; for scale markers with the same value, establish a matching relationship in the detection results on both the left and right sides; let the value of the detection result of the a-th scale digit on the left be... The center point is The value of the detection result of the b-th scale digit on the right is The center point is When the following conditions are met:

[0068] And the positional relationship between the two satisfies:

[0069] Use these as matching numbers on the left and right of the same scale marker; where, This indicates the allowable deviation threshold for numbers with the same scale value in the vertical position of the image; during matching, the color, brightness, or confidence level of the numbers can also be considered to exclude numbers that do not belong to the same scale.

[0070] Assuming the center points of the left and right scale markings with the same value are respectively and Then the scale reference point can be and Determining the midpoint:

[0071] When the first graphic parameter identifier value corresponding to this scale is At this time, the correspondence between the scale reference point and the first graphic parameter can be obtained:

[0072] By sorting multiple scale reference points according to their spatial positions, the scale geometry in the current graphic parameter area can be obtained.

[0073] The specific steps for performing linear interpolation are as follows: assuming two adjacent scale reference points are respectively... and The corresponding first graphic parameters are as follows: and Let the reference mark be ,Will The projection coefficient is calculated when projected onto the scale direction formed by adjacent scale reference points. It can be represented as:

[0074] in, Represents the vector dot product. The L2 norm is used to represent the first graphic parameter of the current frame; linear interpolation is performed between adjacent scales based on the projection coefficients.

[0075] When the interval between adjacent scale parameters is fixed at Δ The distance between adjacent scale reference points is At that time, it can also be expressed as:

[0076] in, Indicates reference mark exist The projection distance in the direction; the above method calculates the first graphic parameter based on the relative position between the reference mark and the adjacent scale reference point, which can reduce the impact of scale number rotation, complex graphic background and partial occlusion on parameter reading.

[0077] The second graphic parameter calculation branch (angle / direction parameters) is specifically: the calculation of the second graphic parameter is based on the angle between the direction vector formed by the graphic key points or scale reference points and the preset reference direction.

[0078] First, construct the orientation vector of the figure; then select two reference points that can represent the rotation direction of the figure. Construct the direction vector of the figure:

[0079] Let the horizontal reference vector of the image be:

[0080] Secondly, calculate the angle between the direction vector and the reference direction; the angle between the graphic direction vector and the horizontal axis of the image can be expressed as:

[0081] in, Used to determine the quadrant of an angle based on the horizontal and vertical components of a vector; if the reference axis direction angle is preset to... Then the second graphical parameter can be expressed as:

[0082] To ensure a consistent output range for the second graphic parameter, angle normalization can be further performed:

[0083] in, This indicates that the parameters are normalized to... Within the range; the sign of the second graphic parameter is determined according to the direction of vector rotation: when the graphic direction vector rotates clockwise relative to the reference direction, the second graphic parameter takes one sign; when it rotates counterclockwise, the second graphic parameter takes the opposite sign. The specific sign rule can be set according to the definition of the graphic parameter area.

[0084] The above calculation transforms the graphic parameters into a geometric solution problem in the target detection result, avoiding direct character recognition and reading of complex graphic parameter regions; after the solution is completed, the first graphic parameters are... With the second graphic parameter The parameters are integrated into the current graphics parameters and used as the output of this step S104.

[0085] Step S105: Based on the text parameter recognition region, the text region is cropped from the image frame to be recognized, and the text region is input into the character recognition model deployed by inference optimization in a batch input manner to obtain the character recognition result.

[0086] In the technical solution provided by step S105 of the present invention, based on the text parameter recognition region obtained by dynamic correction in step S102, image sub-blocks corresponding to each text parameter are accurately cropped from the image frame to be recognized, and sent to the character recognition model in batch input to obtain the character recognition results of each text region, providing text parameter data for subsequent post-processing and field-level fusion.

[0087] Specifically, N dynamically corrected text parameter recognition regions are obtained in the current frame t. (i=1,2,…,N, where N is the number of text parameter regions), each region corresponds to a type of text parameter to be recognized (including numerical parameters, compound text parameters, etc.); the region is identified based on the dynamically corrected text parameters. From the image frame to be identified The corresponding text region images are cropped out one by one from the image. The i-th cropped image is represented as:

[0088] in, This indicates that image cropping is performed based on the identified region. This represents the cropped image of the i-th text region. This indicates a batch input consisting of all text regions in the current frame.

[0089] The N text regions obtained from the cropping are grouped into batch input according to a preset order. The images are fed into a character recognition model deployed using inference optimization. Before being input into the model, the cropped images undergo size normalization, color space conversion, and brightness consistency processing to ensure that different text regions meet the model requirements. This process is applied to batch input images. Perform character recognition to obtain the initial recognition strings for each text region:

[0090] in, Represents a character recognition model. The inference optimization process is represented (including computation graph optimization, adjacent operator fusion, convolution and normalization layer fusion, low-precision computation, storage resource reuse, and execution operator optimization). This indicates the inference engine after deployment using inference optimization methods. This represents the initial recognition string for the i-th text region.

[0091] A batch input method is adopted, enabling the recognition of multiple text parameter regions in the same inference process. Let the total time for region-by-region recognition be... The time required for batch accelerated recognition is Then we have:

[0092] That is, under the same processing conditions, batch input and inference optimization can reduce the overhead of repeated data scheduling and model loading, making the character recognition branch suitable for continuous image frame processing.

[0093] Character recognition models for batch input After performing the recognition, the initial set of recognized strings corresponding to each text region is output. , ,…, }, which is the character recognition result obtained in this step. This character recognition result is still the initial string without post-processing, which may contain misrecognized characters, irrelevant symbols or formatting abnormalities.

[0094] The character recognition model in this invention is a recognition network used to identify characters from text region images obtained after dynamic correction and cropping, and to output character recognition results. It takes text regions cropped from the image frame to be recognized based on text parameter recognition regions as batch input, does not perform text localization on the entire image frame to be recognized, and outputs the initial set of recognition strings corresponding to each text region. The character recognition model is deployed using an inference optimization method, which includes computation graph optimization, adjacent operator fusion, convolution and normalization layer fusion, low-precision computation, storage resource reuse, and execution operator optimization, thereby accelerating the character recognition process and enabling character recognition to adapt to the real-time processing requirements of continuous image frames. The output character recognition results will be fed into subsequent steps for post-processing such as legal character filtering, symbol parsing, numerical formatting, unit conversion, and preset format conversion, and will be fused with graph parameters at the field level to provide a text-type parameter data basis for obtaining the recognition value of the current frame.

[0095] Step S106: Post-process the character recognition results to obtain text parameter results; perform field-level fusion of the text parameter results and the graphic parameters to obtain the recognition value of the current frame.

[0096] In the technical solution provided by step S106 of the present invention, the character recognition result obtained in step S105 is post-processed to obtain a standardized text parameter result, and the text parameter result is fused with the graphic parameter obtained in step S104 at the field level to form the complete instrument display parameter result of the current frame, that is, the current frame recognition value.

[0097] Specifically, the initial string output by the character recognition model It may contain misidentified characters, irrelevant symbols, or abnormal formatting; to ensure that the parameter results conform to the instrument display parameter display rules, post-processing is performed on the string of each character recognition; let the set of legal characters corresponding to the i-th type of instrument display parameter be . The character filtering and format parsing process is represented as follows:

[0098] in, Indicates that it belongs to Remove illegal characters; A function to parse the format of the i-th type of parameter; This represents the numerical result of the parameter displayed by the i-th type of text-based instrument.

[0099] Legal character filtering correspondence For numeric parameters, the set of valid characters is... It can include numbers, decimal points, minus signs, and preset separators; for compound text parameters, characters matching the preset character structure are retained, and parsing is performed according to the preset format; symbol parsing is performed in... Completed in the middle, for signed parameters, the sign is restored according to the string sign position and display area attributes; numeric formatting: in Completed in the middle, for decimal parameters, restore the decimal places according to their display format; unit conversion in The process is completed in the middle, and for parameters with different units, unit conversion is performed according to the unified output format.

[0100] For compound text parameters, if the recognized string is parsed into several components to obtain d, m, and s (such as degrees, minutes, and seconds), the normalized result is expressed as:

[0101] in, This represents the direction sign factor, the value of which can be set according to the parameter display rules; this formula uniformly converts composite text parameters in degree, minute, and second formats into decimal degree formats, achieving preset format conversion. After post-processing, a set of text-type parameter results is obtained. .

[0102] Assume the graphical parameter dataset obtained from step S104 is represented as follows: This includes the first graphic parameter (such as...) (obtained by matching the center points of the left and right scale markers with the same value and linear interpolation) and the second graphic parameter (such as... The angle between the graphic direction vector and the preset reference direction is calculated and normalized to [ ]. 180 180 (Get); the text parameters output by the character recognition branch. Graphical parameters output by the target detection branch Perform field-level fusion; the complete instrument display parameter results for the current frame are as follows:

[0103] in, This represents the complete instrument display parameter result of the current frame (i.e., the current frame identification value) and serves as the input to the continuous frame filtering module. Field-level fusion refers to organizing graphical parameters and text parameters as different fields into the same parameter record according to a preset field order, rather than simply splicing them together. This allows subsequent continuous frame filtering to independently maintain the historical observation window and execute filtering logic for each field (i.e., each specific parameter).

[0104] Step S107: Store the current frame identification value into the historical observation window in the order of shooting time; wherein, the historical observation window is used to store a sequence of continuous frame identification values ​​consisting of the identification values ​​of the current frame and several historical frames; post-processing includes legal character filtering, symbol parsing, numerical formatting, unit conversion and preset format conversion.

[0105] In the technical solution provided by step S107 of the present invention, the current frame identification value obtained after post-processing and completing field-level fusion in step S106 is stored in the historical observation window in the order of shooting time, thereby forming a continuous frame identification value sequence composed of the identification values ​​of the current frame and several historical frames, providing a time-series data basis for subsequent filtering processing of the continuous frame identification value sequence.

[0106] Specifically, the current frame identification value The data is written into the historical observation window in the order of shooting time (i.e., in ascending order of frame number); the historical observation window is a fixed-length sliding buffer structure, and an independent historical observation window is maintained for each instrument display parameter.

[0107] Assume the fusion result of the j-th meter display parameter in frame t is: Its historical observation window is When linear fitting is used for prediction, the parameter values ​​within the historical window satisfy an approximately linear relationship. The fitting coefficients and the predicted value of the current frame are expressed as follows:

[0108] Where M represents the length of the history window; and The fitting coefficient for the j-th parameter; The predicted value is the current frame value; then the deviation between the current frame's recognized value and the predicted value is calculated. Finally, a three-level state machine decision is made, assuming... Let the deviation threshold be the j-th parameter. This is a continuous abnormal state. As a state update condition, the filtered output Represented as:

[0109] If the deviation does not exceed the preset deviation threshold, the current frame identification value is used as the valid output and the historical observation window is updated; if the deviation exceeds the preset deviation threshold and the number of consecutive abnormal frames does not reach the preset number, the previous valid output value or prediction value is output; if the deviation exceeds the preset deviation threshold and the number of consecutive abnormal frames reaches the preset number, the historical observation window is updated with the current frame identification value.

[0110] This mechanism of storing data in chronological order ensures that the historical observation window always contains a sequence of consecutive frame identification values ​​arranged chronologically within the most recent period. Whenever a new frame identification value is written, if the window is full, the oldest frame identification value is discarded, and the window slides to achieve dynamic updates of the historical observation window.

[0111] Step S108: Filter the sequence of consecutive frame recognition values ​​and output the parameter recognition results.

[0112] In the technical solution provided by step S108 of the present invention, the continuous frame identification value sequence, which is stored by the historical observation window in step S107 and consists of the identification values ​​of the current frame and several historical frames, is filtered. Trend prediction is performed based on the historical observation window, the deviation between the identification value and the predicted value of the current frame is compared, and the output result of the current frame is determined based on the deviation and the number of consecutive abnormal frames, so as to output a highly stable parameter identification result.

[0113] Specifically, after filtering, structured parameter records are generated according to a preset field order. The structured output data is organized using a field-based approach, including the original fusion result, the filtered result, validity identifier, frame identifier, and time stamp. Each structured result is represented as follows:

[0114] in, This is the original fusion result of the current frame (i.e., the complete instrument display parameter result of the current frame obtained in step S106). This is the filtered result (i.e., the parameter identification result). This serves as a field validity flag. For frame numbering, This provides time information. The structured results can be used for real-time display, file saving, or transfer to external systems.

[0115] The method described in this embodiment will be further described below.

[0116] As an optional embodiment, step S104, the graphic parameters include a first graphic parameter and a second graphic parameter. Based on the graphic parameter region detection results, the graphic parameters are calculated, including: filtering scale markers located on both sides of the graphic parameter detection region in the graphic parameter region detection results that have the same numerical attributes; calculating the coordinates of the center point of the scale markers to generate scale reference points; extracting reference markers from the graphic parameter region detection results and calculating the positional offset of the reference markers relative to adjacent scale reference point data; obtaining the first graphic parameter through a linear interpolation algorithm based on the proportional relationship between the positional offset and the preset scale spacing data; extracting the coordinates of graphic key points or scale reference points from the graphic parameter region detection results, constructing direction vector data, calculating the angle value between the direction vector data and the preset reference direction data, and assigning a sign attribute based on the quadrant in which the vector is located to obtain the normalized second graphic parameter; and integrating the first graphic parameter and the second graphic parameter into a single graphic parameter.

[0117] In this embodiment, the graphic parameters are specifically decomposed into a dual-branch solution of a first graphic parameter of position or displacement and a second graphic parameter of angle or direction. Through two independent geometric solution paths, the different types of parameter reading needs in the instrument graphic indication area are covered.

[0118] Specifically, the algorithm filters out scale markers with the same numerical attributes located on both sides of the graphic parameter detection area in the graphic parameter area detection results. For the matching criteria, center point calculation, and specific formulas for generating scale reference points for scale markers with the same value, please refer to the formulas explained in step S104 above. After calculating the scale reference points, the algorithm extracts the reference markers from the graphic parameter area detection results and calculates the positional offset of the reference markers relative to adjacent scale reference point data. Based on the proportional relationship between this positional offset and the preset scale spacing data, the first graphic parameter is obtained through a linear interpolation algorithm.

[0119] The coordinates of key points or scale reference points in the graphic parameter area detection results are extracted to construct direction vector data. The scale reference points are the reference points generated in the aforementioned first graphic parameter calculation. Both can be used as reference points for constructing the direction vector. The angle between the direction vector data and the preset reference direction data is calculated, and a sign attribute is assigned based on the quadrant in which the vector is located to obtain the normalized second graphic parameter.

[0120] The first and second graphic parameters are integrated into a single graphic parameter, which is the final output of step S104.

[0121] As an optional embodiment, step S108, legal character filtering, includes: establishing a set of legal characters for the preset formats of different instrument display parameters; for numerical parameters, retaining numeric characters, decimal points, minus signs, and preset separators, and removing other interfering characters; for composite format text parameters using degrees, minutes, and seconds, converting them into floating-point data format of decimal degrees according to preset conversion rules.

[0122] In this embodiment, the set of legal characters is established as a preset format for different instrument display parameters, and a corresponding set of legal characters A is established. i The character filtering process is described in the aforementioned formula (38); the filtering rule for numeric parameters is that for numeric parameters, their legal character set A... i This includes numeric characters, decimal points, minus signs, and preset separators. During filtering, characters belonging to this set are retained, while other interfering characters are removed.

[0123] The filtering and conversion rules for composite text parameters are as follows: for composite text parameters using degree, minute and second formats, retain the characters that match the preset character structure, and convert them into floating-point data format of decimal degrees according to the preset conversion rules. For specific conversion formulas, please refer to the aforementioned formula (39).

[0124] This optional embodiment ensures that the filtered character sequence can accurately enter the subsequent symbol parsing, numerical formatting, unit conversion, and preset format conversion stages by establishing differentiated sets of legal characters for different format parameters.

[0125] As an optional embodiment, step S108, filtering the continuous frame identification value sequence and outputting parameter identification results, includes: if the deviation between the current frame identification value and the predicted value does not exceed a preset deviation threshold, determining the current frame identification value as the output result and updating the historical observation window; if the deviation exceeds the preset deviation threshold and the number of consecutive abnormal frames does not reach a preset number, maintaining the previous valid output value unchanged and updating the number of consecutive abnormal frames; wherein, the number of consecutive abnormal frames is the cumulative value of the number of frames in the continuous frame identification value sequence where the deviation exceeds the standard; if the deviation exceeds the preset deviation threshold and the number of consecutive abnormal frames reaches a preset number, determining the current frame identification value as the output result and using the current frame identification value to update the historical observation window and the number of consecutive abnormal frames.

[0126] In this embodiment, trend prediction is performed based on the sequence of continuous frame identification values ​​in the historical observation window to obtain the predicted value of the current frame; the deviation between the current frame identification value and the predicted value is calculated. The historical observation window is used to store a sequence of consecutive frame identification values, including the identification values ​​of the current frame and several historical frames. The formula logic for trend prediction is shown in the aforementioned formula (41).

[0127] The process of determining the third-level state machine is as follows: Level 1 (Normal Frame Judgment) means that if the deviation between the current frame identification value and the predicted value does not exceed the preset deviation threshold, the current frame identification value is determined as the output result, and the data in the historical observation window is updated (that is, the current frame identification value is stored in the historical observation window in the order of shooting time to participate in subsequent trend prediction); this situation corresponds to the number of consecutive abnormal frames being cleared to zero.

[0128] The second level (transient abnormal frame determination) means that if the deviation exceeds the preset deviation threshold and the number of consecutive abnormal frames does not reach the preset number, the previous valid output value is maintained unchanged, and only the consecutive abnormal count is updated. The technical significance of this determination is that the single frame recognition result may be affected by image blurring, local reflection, occlusion or model false detection, resulting in instantaneous abnormal values. By maintaining the previous valid output value, the pollution of the output sequence by short-term abnormal values ​​is effectively suppressed, and parameter sequence jitter is avoided. The previous valid output value refers to the parameter value that was most recently recognized and output by the system before the current frame.

[0129] The third level (continuous anomalies are identified as real changes) means that if the deviation exceeds a preset deviation threshold and the number of consecutive abnormal frames reaches a preset number, the current frame identification value is determined as the output result, and the historical observation window and the number of consecutive abnormal frames are updated using the current frame identification value. Specifically, the historical observation window is reset using the current frame identification value (the current frame identification value is used as the first data to reconstruct the sequence of consecutive frame identification values), and the number of consecutive abnormal frames is cleared to zero. The technical significance of this determination is that when the anomaly lasts for multiple frames, it indicates that this may be a real parameter change rather than a misidentification. At this time, the current frame identification value is accepted and the historical state is updated to avoid long-term lag in the output result and to respond to real parameter changes in a timely manner.

[0130] An embodiment of the present invention provides an instrument display parameter recognition system based on the fusion of target detection and character recognition, comprising: The image acquisition module is used to acquire images of the instrument display interface and preprocess the images to obtain the image frame to be identified. The preprocessing includes image format conversion, resolution normalization, and image denoising. The image frame to be identified output by the image acquisition module will be sent to the region correction module as a unified input for subsequent homography mapping and region correction. For the specific operation of preprocessing and coordinate normalization formula, please refer to the relevant formula in step S101 above.

[0131] The region correction module is the core module for eliminating installation deviations, vibration drift, and perspective distortion. Specifically, this module includes: calculating a homography mapping matrix based on the correspondence between feature points between a reference image and the image frame to be identified; transforming multiple boundary points of the preset recognition region into the image frame to be identified using the homography mapping matrix; and determining the dynamically corrected graphic parameter detection region and text parameter recognition region based on the transformed boundary points. The feature points include corner points, boundary points, stable texture points, or preset marker points. The formulas for calculating the homography mapping matrix, transforming boundary points, and determining the dynamically corrected recognition region are detailed in step S102 above.

[0132] The region correction module is also used to establish a feature point correspondence between the reference image and the image frame to be identified based on corner points, boundary points, stable texture points, or preset marker points. This correspondence is both the basis for calculating the homography mapping matrix and the criterion for triggering region updates: when the feature point mapping error or key point offset exceeds the preset threshold, the region correction module re-executes feature point extraction and homography matrix calculation to ensure that the dynamically corrected graphic parameter detection area and text parameter recognition area are always aligned with the actual instrument display content of the current frame.

[0133] The dynamically corrected graphic parameter detection area output by the region correction module is sent to the graphic parameter recognition module, and the dynamically corrected text parameter recognition area is sent to the character recognition module, thereby achieving input alignment for dual-branch parallel processing.

[0134] Figure 2 This is a schematic diagram of an instrument display interface before identification area correction according to an embodiment of the present invention, as shown below. Figure 2 As shown in the figure, the red rectangles indicate the initial projection positions of the system's preset graphic parameter detection area and text parameter recognition area on the original image. The central semi-circular area (large red box) is the graphic parameter detection area, which includes posture indicators and scales. The surrounding small rectangles (small red boxes) are the text parameter recognition areas, used to capture surrounding values. In uncorrected real-world scenarios, due to factors such as camera installation angle and perspective distortion, these preset areas often appear tilted or not in a normal viewing state in the original image. Subsequent steps will calculate the homography mapping matrix based on the correspondence between feature points of the reference image and the image frame to be recognized, and dynamically correct these initial areas to a standard normal viewing state to eliminate distortion interference and ensure the accuracy of subsequent graphic detection and character recognition.

[0135] Figure 3 This is a schematic diagram of an instrument display interface after recognition region correction according to an embodiment of the present invention. Based on the homography mapping matrix calculated above, the system dynamically transforms the preset recognition region into the image frame to be recognized; for example... Figure 3As shown, perspective distortion is effectively eliminated, and the central graphic parameter detection area (the blue-filled area in the figure) is corrected into a regular semi-circular shape, containing horizontal attitude indicators and scale lines, presenting a normal viewing state without geometric distortion. Simultaneously, the surrounding text parameter recognition areas are also precisely aligned, accurately selecting "1.6" in the upper left corner, "0.59" in the lower left corner, "127" in the upper right corner, and "6.499" in the lower right corner. This figure visually demonstrates the technical effect of the area correction module: through dynamic coordinate transformation, the initial tilted and irregular detection box is transformed into a standard normal viewing area that perfectly overlaps with the instrument's physical display, providing a high-quality input foundation for subsequent dual-branch parallel recognition.

[0136] The graphic parameter recognition module inputs the graphic parameter detection area into the target detection model, detects the reference marker, graphic key points, and scale markers, and obtains the graphic parameter area detection result. The target detection model of this graphic parameter recognition module uses the dynamically corrected graphic parameter detection area image as the only input, and simultaneously detects three types of graphic elements: reference marker, graphic key points, and scale markers, and outputs the graphic parameter area detection result including category, detection box position, and confidence level. This detection result is sent to the graphic parameter solution module as the element basis for geometric solution. For the specific process of target detection, detection box representation, and center point calculation formula, please refer to the relevant formulas in step S103 above.

[0137] The graphic parameter calculation module calculates graphic parameters based on the graphic parameter region detection results. This module specifically includes a first graphic parameter calculation branch and a second graphic parameter calculation branch: the first graphic parameter is obtained by selecting scale markers with the same numerical attributes on both sides of the graphic parameter detection region, calculating the center point coordinates of the scale markers with the same value and generating scale reference points, extracting the reference markers and calculating their projection distances relative to adjacent scale reference points, and obtaining the first graphic parameter through a linear interpolation algorithm based on the proportional relationship between the projection distance and the distance between adjacent scale reference points; the relevant formulas are described in step S104 above.

[0138] The second graphic parameter is obtained by extracting key points or scale reference points of the graphic to construct a direction vector, calculating the angle between the direction vector and the preset reference direction, and assigning a sign attribute based on the quadrant in which the vector is located, thus obtaining the normalized second graphic parameter; the relevant formulas are described in the aforementioned step S104.

[0139] The graphics parameter calculation module sends the integrated graphics parameter dataset to the data processing and output module for subsequent field-level fusion.

[0140] The character recognition module identifies text regions based on text parameters. It crops text regions from the image frame to be recognized and inputs these regions in batches to a character recognition model deployed using inference optimization, obtaining the character recognition results. This module only performs character recognition on the dynamically corrected, cropped text regions, without performing text localization on the entire image frame, thus eliminating interference from the graphic background. The character recognition model is deployed using inference optimization (including computation graph optimization, adjacent operator fusion, convolution and normalization layer fusion, low-precision computation, storage resource reuse, and execution operator optimization) to meet the real-time processing requirements of continuous frames. The relevant batch input and recognition formulas are described in step S105 above. The character recognition module then sends the character recognition results to the data processing and output module.

[0141] The data processing and output module is specifically used for: post-processing and field-level fusion: post-processing the character recognition results to obtain text parameter results; and fusion of the text parameter results with the graphic parameters at the field level to obtain the current frame recognition value. Post-processing includes legal character filtering, symbol parsing, numerical formatting, unit conversion, and preset format conversion. Legal character filtering establishes a set of legal characters for preset formats of different instrument display parameters. For numerical parameters, it retains numeric characters, decimal points, minus signs, and preset separators. For composite text parameters using degrees, minutes, and seconds formats, it converts them to decimal floating-point data format according to preset conversion rules. The formulas for post-processing and field-level fusion are described in step S106 above.

[0142] The data processing and output module is also used for constructing continuous frame sequences: the current frame identification value is stored in the historical observation window in the order of shooting time; the historical observation window is used to store the continuous frame identification value sequence consisting of the identification values ​​of the current frame and several historical frames.

[0143] The data processing and output module is also used for continuous frame filtering and output: filtering the continuous frame identification value sequence, determining and outputting the output result of the current frame based on the continuous frame identification value sequence. The filtering process specifically includes: trend prediction based on the historical observation window (see the formula (41) above), comparing the deviation between the current frame identification value and the predicted value, and determining the output result of the current frame based on the deviation and the number of consecutive abnormal frames: if the deviation does not exceed the preset deviation threshold, the current frame identification value is determined as the output result and the historical observation window is updated; if the deviation exceeds the preset deviation threshold and the number of consecutive abnormal frames does not reach the preset number, the previous valid output value is kept unchanged and the consecutive abnormal count is updated; if the deviation exceeds the preset deviation threshold and the number of consecutive abnormal frames reaches the preset number, the current frame identification value is determined as the output result, and the historical observation window and the consecutive abnormal count are reset using the current frame identification value; the complete formula logic for filtering determination is shown in the formula (42) above.

[0144] After filtering is completed, the data processing and output module generates a structured parameter record according to the preset field order (see formula (43) above). The structured parameter record includes the original fusion result, the filtered result, the validity identifier, the frame identifier and the time stamp. It can be output through the data interface, saved locally or provided to the upper layer application for device status monitoring, data recording and auxiliary analysis.

[0145] Figure 4 This is a schematic diagram of structured parameter output according to an embodiment of the present invention. After completing the post-processing of the character recognition results (including legal character filtering, symbol parsing, numerical formatting, unit conversion, and preset format conversion) and performing field-level fusion with the graphic parameters, the data processing and output module generates the original fusion result of the current frame. Furthermore, the system filters the sequence of identification values ​​for consecutive frames based on historical observation windows and outputs the filtered result. And generate the corresponding field validity flags. Frame number and time information This constructs a complete structured parameter record. The original fusion result is included. Record the initial recognition values ​​of the current frame after field-level fusion, preserving the most original perceptual information; the filtered result Records stable values ​​after three-level state machine filtering to eliminate transient interference; field validity flag. Record the quality status of the current frame data, used to identify whether the frame data is valid or in an abnormal / lost state; frame number. Record the unique sequence number of the current frame for timing tracking; timing information Records the physical timestamp of the current frame data acquisition. This structured parameter record can be transmitted in real time to the local storage unit for data archiving through a preset data interface, or pushed to upper-layer applications to drive device status monitoring and data analysis.

[0146] In this embodiment of the invention, an image of the instrument display interface is acquired and preprocessed to obtain an image frame to be identified. The preprocessing includes image format conversion, resolution normalization, and image denoising. Based on the correspondence between feature points of the reference image and the image frame to be identified, a homography mapping matrix is ​​calculated. Multiple boundary points of the preset recognition region are transformed into the image frame to be identified using the homography mapping matrix. Based on the transformed boundary points, the dynamically corrected graphic parameter detection region and text parameter recognition region are determined. Feature points include corner points, boundary points, stable texture points, or preset marker points. The graphic parameter detection region is input into the target detection model to detect the reference marker, graphic key points, and scale markers, obtaining the graphic parameter region detection result. Based on the graphic parameter region... The system performs the following steps: First, it calculates graphic parameters based on the domain detection results. Then, it identifies the text region from the image frame to be recognized, inputting the text region in batches to a character recognition model deployed using inference optimization to obtain character recognition results. Next, it performs post-processing on the character recognition results to obtain text parameter results. Finally, it fuses the text parameter results with the graphic parameters at the field level to obtain the current frame's recognition value. The current frame's recognition value is stored in the historical observation window in chronological order of capture time. The historical observation window stores a sequence of consecutive frame recognition values, including the current frame and several historical frames. Post-processing includes legal character filtering, symbol resolution, numerical formatting, unit conversion, and preset format conversion. Finally, it filters the consecutive frame recognition value sequence and outputs the parameter recognition results.

[0147] This invention solves the problems in existing instrument visual recognition caused by installation deviations of image acquisition devices, perspective distortion of the image, misalignment of recognition areas, difficulty in stably reading graphic parameters through character recognition, susceptibility of whole-image character recognition to interference from graphic background, and easy jumps in single-frame recognition results. It eliminates recognition misalignment caused by installation deviations and perspective distortion by dynamically correcting the graphic parameter detection area and text parameter recognition area through homography mapping. It overcomes the limitations of conventional OCR in graphic parameter recognition by simultaneously extracting reference markers, graphic key points, and scale markers using a target detection model to calculate graphic parameters. By batch character recognition of the corrected text area and field-level fusion with graphic parameters, it greatly reduces computational overhead while avoiding graphic noise interference. By filtering the continuous frame recognition value sequence based on historical observation windows, it accurately identifies instantaneous outliers, suppressing output jitter while responding promptly to real parameter changes, thereby outputting continuous and stable structured instrument display parameters.

[0148] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0149] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0150] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between units or modules may be electrical or other forms.

[0151] The units described as separate components may or may not be physically separate. Similarly, the components shown as units may or may not be physical units; they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0152] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a first processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0153] The above are merely preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for recognizing instrument display parameters based on the fusion of target detection and character recognition, characterized in that, include: Acquire an image of the instrument display interface, and preprocess the image to obtain an image frame to be identified; wherein, the preprocessing includes image format conversion, resolution normalization, and image denoising; Based on the correspondence between feature points between the reference image and the image frame to be identified, a homography mapping matrix is ​​calculated. Multiple boundary points of the preset recognition region are transformed into the image frame to be identified through the homography mapping matrix. Based on the transformed boundary points, the dynamically corrected graphic parameter detection region and text parameter recognition region are determined. The feature points include corner points, boundary points, stable texture points, or preset marker points. The graphic parameter detection area is input into the target detection model to detect the reference marker, graphic key points and scale markers, and obtain the graphic parameter area detection result; Based on the region detection results of the graphic parameters, the graphic parameters are calculated; Based on the text parameter recognition region, the text region is cropped from the image frame to be recognized, and the text region is input into the character recognition model deployed by inference optimization in a batch input manner to obtain the character recognition result; The character recognition result is post-processed to obtain the text parameter result; the text parameter result is then fused with the graphic parameter at the field level to obtain the current frame recognition value. The current frame identification value is stored in the historical observation window in chronological order of shooting time; wherein, the historical observation window is used to store a sequence of continuous frame identification values ​​consisting of the identification values ​​of the current frame and several historical frames; the post-processing includes legal character filtering, symbol parsing, numerical formatting, unit conversion and preset format conversion; The continuous frame recognition value sequence is filtered to output the parameter recognition result.

2. The method according to claim 1, characterized in that, The graphic parameters include a first graphic parameter and a second graphic parameter. The calculation of the graphic parameters based on the region detection results of the graphic parameters includes: Filter the scale markers with the same numerical attributes located on both sides of the graphic parameter detection area in the graphic parameter area detection results; Calculate the coordinates of the center point of the scale mark to generate a scale reference point; Extract the reference identifier from the detection results of the graphic parameter area, and calculate the positional offset of the reference identifier relative to the adjacent scale reference point data; Based on the proportional relationship between the position offset and the preset scale spacing data, the first graphic parameters are obtained through a linear interpolation algorithm; The coordinates of key points or scale reference points of the graphics are extracted from the detection results of the graphics parameter area, direction vector data is constructed, the angle between the direction vector data and the preset reference direction data is calculated, and a sign attribute is assigned based on the quadrant in which the vector is located, so as to obtain the normalized second graphics parameter. The first graphic parameter and the second graphic parameter are integrated into the graphic parameter.

3. The method according to claim 1, characterized in that, The filtering of legal characters includes: Establish a set of legal characters for the preset formats of different instrument display parameters; For numerical parameters, retain numeric characters, decimal points, minus signs, and preset separators, and remove other interfering characters; For text parameters in composite format using degrees, minutes, and seconds, convert them into floating-point data format in decimal degrees according to preset conversion rules.

4. The method according to claim 1, characterized in that, The filtering process of the continuous frame recognition value sequence to output parameter recognition results includes: If the deviation between the current frame identification value and the predicted value does not exceed the preset deviation threshold, the current frame identification value is determined as the output result, and the historical observation window is updated. If the deviation exceeds the preset deviation threshold and the number of consecutive abnormal frames does not reach the preset number, the previous valid output value is maintained unchanged, and the number of consecutive abnormal frames is updated; wherein, the number of consecutive abnormal frames is the cumulative value of the number of frames in the consecutive frame identification value sequence where the deviation exceeds the standard. If the deviation exceeds the preset deviation threshold and the number of consecutive abnormal frames reaches a preset number, the current frame identification value is determined as the output result, and the historical observation window and the number of consecutive abnormal frames are updated using the current frame identification value.

5. A system for recognizing instrument display parameters based on the fusion of target detection and character recognition, characterized in that, include: The image acquisition module is used to acquire images of the instrument display interface and preprocess the images of the instrument display interface to obtain image frames to be identified; wherein, the preprocessing includes image format conversion, resolution normalization and image denoising; The region correction module is used to calculate a homography mapping matrix based on the correspondence between feature points between the reference image and the image frame to be identified, and to transform multiple boundary points of the preset recognition region into the image frame to be identified through the homography mapping matrix. Based on the transformed boundary points, the dynamically corrected graphic parameter detection region and text parameter recognition region are determined. The feature points include corner points, boundary points, stable texture points, or preset marker points. The graphic parameter recognition module is used to input the graphic parameter detection area into the target detection model, detect the reference mark, graphic key points and scale mark, and obtain the graphic parameter area detection result; The graphic parameter calculation module is used to calculate graphic parameters based on the graphic parameter region detection results; The character recognition module is used to identify the text region based on the text parameters, crop out the text region from the image frame to be recognized, and input the text region into the character recognition model deployed by inference optimization in a batch input manner to obtain the character recognition result; The data processing and output module is used to post-process the character recognition results to obtain text parameter results; to perform field-level fusion of the text parameter results and the graphic parameters to obtain the current frame recognition value; to store the current frame recognition value in the historical observation window in the order of shooting time; wherein, the historical observation window is used to store a continuous frame recognition value sequence consisting of the recognition values ​​of the current frame and several historical frames; the post-processing includes legal character filtering, symbol parsing, numerical formatting, unit conversion and preset format conversion; to perform filtering processing on the continuous frame recognition value sequence, and to determine and output the output result of the current frame based on the continuous frame recognition value sequence.

6. The system according to claim 5, characterized in that, The region correction module is also used for: Based on corner points, boundary points, stable texture points, or preset marker points, establish the correspondence between feature points between the reference image and the image frame to be identified.

Images