Electronic atomizer sensing signal data reading method based on image recognition

Abstract

The present application relates to the technical field of image recognition and industrial vision detection, and discloses an electronic atomizer sensing signal data reading method based on image recognition, which comprises the following steps: collecting panoramic images through an industrial camera array and simultaneously locating LED indicator lights and tester screens by using a double-class YOLO detection network; estimating and compensating the interference of LED halos on screen areas based on a point spread function; unsupervisedly analyzing screen layouts through multi-scale Canny edge detection and Hough transformation; respectively performing OCR recognition and skeleton extraction on screen numerical reading areas and waveform areas; after adaptive brightness normalization and optical crosstalk compensation are performed on LED sub-images, extracting timing code matching sensing signals; and finally cross- verifying double-source data and outputting batch reading results.

Description

Technical Field

[0001] This invention relates to the fields of image recognition and industrial visual inspection technology, and more specifically, to a method for reading sensor signal data of electronic atomizers based on image recognition. Background Technology

[0002] At the batch quality inspection station at the end of the electronic atomizer production line, dozens of atomizers are simultaneously placed on test fixtures and powered on. The LED indicator lights of each atomizer display the sensor signal status in a color and brightness encoding manner. At the same time, each atomizer is connected to the test instrument next to it via a cable to display precise measurement data.

[0003] Existing methods acquire video frame sequences of the LED indicator area of ​​a single electronic atomizer using a camera device, identify color and brightness through HSV color space conversion, and generate LED timing codes by analyzing the flicker frequency and color switching sequence along the time axis. These codes are then matched with a sensor signal mapping table to obtain sensor signal data. For the tester screen, a pre-trained U-Net segmentation network is used to divide the waveform display area and the numerical reading area, and OCR recognition and waveform tracking are performed respectively to read the instrument measurement data.

[0004] However, in the scenario of panoramic image acquisition at batch quality inspection stations, the existing methods have the following technical problems: the high brightness of the active LED indicator light generates a large-scale halo on the panoramic image, forming reflective flares that interfere with the screen area of ​​adjacent testing instruments, resulting in a large number of false edges and misjudgments of the screen area edge detection, causing the testing instrument screen data reading to fail; the backlight brightness change of the testing instrument screen interferes with the brightness analysis of adjacent LED indicator lights, and the uniform HSV brightness threshold cannot simultaneously adapt to the characteristics of two types of light sources, namely active LED indicator lights and backlight transmissive LCD screens, resulting in inaccurate determination of the brightness status of LED indicator lights; the screen layouts of various instruments vary greatly, and the layout may change after firmware upgrades. When the U-Net segmentation network encounters a new layout not included in the training set, the segmentation accuracy drops sharply, making it unable to quickly adapt to instrument replacement or firmware upgrade scenarios. These problems prevent the existing methods from simultaneously completing the encoding and recognition of all atomizer LED indicator lights and the reading of testing instrument screen data from a unified vision acquisition system, making it difficult to meet the batch sensor signal data reading requirements of high-speed production lines. Summary of the Invention

[0005] This invention provides a method for reading sensor signal data of electronic atomizers based on image recognition, which solves the technical problems in the process of reading sensor signal data of batch quality inspection stations, such as LED indicator halo interference tester screen image, difficulty in automatic analysis of screen layout, and lack of multi-source data fusion verification.

[0006] This invention discloses a method for reading sensor signal data of electronic atomizer based on image recognition, including: acquiring panoramic images of batch quality inspection stations, simultaneously outputting the bounding box coordinates and station number labels of each LED indicator and the bounding box coordinates and instrument position labels of each tester screen through a dual-class target detection network, and generating an LED positioning index table and a screen positioning index table. Based on the LED positioning index table and the screen positioning index table, establish the spatial adjacency relationship between the LED indicator and the tester screen. Use the point spread function to calculate the halo intensity value of each LED indicator for each pixel position in the adjacent tester screen area. Accumulate the halo intensity values ​​of all adjacent LED indicators pixel by pixel to generate the halo contribution matrix. Subtract the halo contribution matrix pixel by pixel from the original image of the tester screen area to generate the halo-compensated screen sub-image. Multi-scale edge detection and line detection are performed on the halo-compensated screen sub-image to divide rectangular sub-regions. Based on the content type discrimination features of each rectangular sub-region, the rectangular sub-regions are classified into numerical reading regions, waveform display regions, status icon regions, or decorative regions. Data reading is performed on each classified sub-region to generate structured screen data reading results. Based on the LED positioning index table, each LED indicator independent sub-image is cropped, adaptive brightness normalization and optical crosstalk compensation are performed on each LED indicator independent sub-image, the LED timing encoding sequence is extracted along the time axis of the video frame sequence, and the LED sensing signal data reading result is obtained by matching with the encoding and sensing signal mapping table. The LED sensor signal data reading results corresponding to the same atomizer are paired, correlated, and cross-validated with the structured screen data reading results. Weighted fusion or anomaly marking is performed based on the absolute deviation of the dual-source parameter values, and a batch sensor signal data reading report is output.

[0007] Furthermore, the calculation of the halo intensity value of each LED indicator at each pixel position in the adjacent tester screen area using the point spread function includes: Calculate the Euclidean distance between the center coordinates of all LED indicator bounding boxes and the boundaries of all tester screen bounding boxes, filter the pairing relationships of LED indicators and tester screens with a distance less than a preset adjacency distance threshold, and generate a spatial adjacency table; The point spread function is calculated as follows: the halo intensity value of a single LED indicator at a certain pixel position in the tester screen area is equal to the peak brightness of the LED indicator sub-image divided by the sum of the attenuation exponent obtained by adding the ratio of the Euclidean distance from the pixel position to the center of the LED indicator bounding box to the halo spread scale parameter, wherein the halo spread scale parameter and the attenuation exponent are obtained in advance based on the LED indicator bounding box size and camera optical parameters. The halo contribution matrix is ​​generated by accumulating the halo intensity values ​​of all adjacent LED indicators within the same tester screen area pixel by pixel. The corresponding values ​​of the halo contribution matrix are then subtracted from the original pixel values ​​of the tester screen area pixel by pixel, and the compensated pixel values ​​are truncated to a non-negative range.

[0008] Furthermore, the process of performing multi-scale edge detection and line detection on the halo-compensated screen sub-image to divide it into rectangular sub-regions, and classifying the rectangular sub-regions based on the content type discrimination features of each rectangular sub-region, includes: Canny edge detection is performed on the halo-compensated screen sub-image at low, medium, and high threshold levels to extract three edge maps. The three edge maps are then logically ORed and merged to generate a screen content structure line graph. Based on Hough transform, horizontal and vertical line segments are detected from the screen content structure line graph. Valid partition boundary lines are selected according to the condition that the line segment length is greater than the corresponding side length of the screen sub-image by a preset ratio. Based on the intersection and endpoint coordinates of the partition boundary lines, the screen sub-image is divided into several rectangular sub-regions. Content type discrimination features are extracted for each rectangular sub-region. These features include text density features, curve features, and icon features. The text density features are obtained by performing binarization and connected component analysis on the rectangular sub-region, calculating the ratio of the total area of ​​the connected components to the area of ​​the rectangular sub-region, as well as the average aspect ratio of the connected components. The curve features are obtained by statistically analyzing the proportion of the area of ​​slender connected components within the rectangular sub-region whose ratio of the major axis to the minor axis is greater than the slenderness ratio threshold. The icon features are obtained by statistically analyzing the mean roundness and standard deviation of the area of ​​compact connected components within the rectangular sub-region whose ratio of the major axis to the minor axis is less than the compactness ratio threshold. The classification is performed using a rule-based decision tree with the following priority: First, it is determined whether the text density feature meets the condition of a dense text region. If it does, it is classified as a numerical reading region. Second, it is determined whether the curve feature meets the condition of a dense curve region. If it does, it is classified as a waveform display region. Third, it is determined whether the icon feature meets the condition of a dense icon region. If it does, it is classified as a status icon region. If none of the above conditions are met, it is classified as a decorative region.

[0009] Furthermore, before performing multi-scale Canny edge detection, contrast-limited adaptive histogram equalization is performed on the halo-compensated screen sub-image to enhance the edge information of weak contrast areas in the screen sub-image.

[0010] Furthermore, the step of performing data reading for each category sub-region includes: Connectivity analysis is performed on the numerical reading region to obtain the character size statistical distribution by statistically analyzing the height values ​​of the bounding rectangles of each connected region. Based on the median of the character size statistical distribution, a matching OCR recognition model granularity is searched in a preset OCR model granularity mapping table. The OCR recognition model with the matching granularity is used to perform character recognition on the numerical reading region to obtain the initial numerical recognition result. The physical quantity unit symbol adjacent to the numerical string space is searched in the initial numerical recognition result. The initial numerical recognition result is post-corrected based on the reasonable range of the numerical range corresponding to the physical quantity unit symbol, and the numerical reading and physical quantity label are output. The post-correction is as follows: when the initial numerical recognition result exceeds the reasonable range of the numerical range, the character with the lowest confidence in the initial numerical recognition result is replaced with a candidate character, and the recognition result that falls into the reasonable range of the numerical range after replacement is selected as the corrected numerical reading. The text connected components are searched within a strip of preset width at the top, bottom, left, and right edges of the waveform display area. The OCR recognition model is used to perform character recognition on the text connected components to obtain scale values. Based on the correspondence between the pixel positions of each scale value in the edge strip area and the scale values, a pixel-physical coordinate mapping matrix is ​​established through linear interpolation. Morphological skeleton extraction is performed on the waveform curve in the waveform display area to obtain a curve skeleton with a single pixel width. The pixel coordinate sequence of the curve skeleton is converted into a physical coordinate sequence using the pixel-physical coordinate mapping matrix to generate a numerical time series data sequence. Based on the spatial adjacency of each rectangular sub-region within the screen sub-image, the numerical readings are semantically associated with physical quantity labels and numerical time-series data sequences to generate structured screen data reading results.

[0011] Furthermore, before performing morphological skeleton extraction on the waveform display area, color clustering separation processing is also performed on the waveform display area. The color clustering separation processing adopts the K-means clustering algorithm. The input of the K-means clustering algorithm is the RGB value of each pixel in the waveform display area, and the output of the K-means clustering algorithm is the cluster label of each pixel. After separating the waveform curve pixels from the background pixels, morphological skeleton extraction is performed only on the waveform curve pixels.

[0012] Furthermore, the adaptive brightness normalization and optical crosstalk compensation performed on each LED indicator's independent sub-image includes: Calculate the brightness histogram of the independent sub-image of the LED indicator, and calculate the average brightness value as the exclusive brightness benchmark value based on the set of pixels whose brightness value is greater than the background brightness threshold in the brightness histogram; The tester screen boundary region adjacent to the LED indicator is obtained based on spatial adjacency relationship. The average brightness value of the tester screen boundary region is calculated as the ambient light component. The ambient light component is subtracted from the brightness of each pixel in the independent sub-image of the LED indicator. The normalization coefficient is calculated based on the exclusive brightness reference value. The normalization coefficient is equal to the ratio of the preset reference brightness value to the exclusive brightness reference value. The brightness of each pixel after removing the ambient light component is multiplied by the normalization coefficient to generate an independent sub-image of the LED indicator after brightness normalization. For regions where there is spatial overlap between the bounding boxes of adjacent LED indicator lights, the pixel brightness value sequence is sampled along the line connecting the centers of the two LED indicator lights within the overlapping region. The pixel brightness value sequence is decomposed according to the hue weight of each LED indicator light. The diffusion scale parameter and attenuation index in the point spread function are fitted using the least squares method. The halo contribution of each LED indicator light to the adjacent LED indicator light region is calculated, and the halo contribution is subtracted pixel by pixel from the pixels of the adjacent LED indicator light region. The hue weight is decomposed as follows: the hue weight of a sampled pixel in the overlapping area belonging to the first LED indicator is equal to the hue ring angle distance between the hue value of the sampled pixel and the main hue of the second LED indicator divided by the sum of the hue ring angle distances between the hue value of the sampled pixel and the main hues of the two LED indicators; the hue weight belonging to the second LED indicator is equal to the hue ring angle distance between the hue value of the sampled pixel and the main hue of the first LED indicator divided by the sum of the above angle distances.

[0013] Furthermore, when calculating the exclusive brightness reference value, a bimodal distribution is also performed on the brightness histogram. When a bimodal distribution is detected, the average brightness value corresponding to the higher brightness peak is taken as the exclusive brightness reference value.

[0014] Furthermore, the extraction of the LED timing-coded sequence along the time axis of the video frame sequence includes: For each LED indicator, extract the hue channel value and luminance channel value of the HSV color space in each frame of the image to generate hue, time series and luminance, time series; After performing cubic spline interpolation resampling on the brightness and time series to normalize the unequally spaced sampling times into an unequally spaced time series, peak detection is performed, the time interval between adjacent brightness peaks is calculated, and the flicker frequency is obtained by taking the reciprocal of the arithmetic mean of multiple adjacent peak time intervals. A step change detection is performed on the hue and time series. The step change detection is to calculate the hue difference between adjacent frames in the hue and time series. When the absolute value of the hue difference between adjacent frames exceeds the preset hue step threshold, it is determined that a color switch has occurred. The switching time and the average hue value of the stable frame segment after the switch are recorded to generate a color switch sequence. The flashing frequency and color switching sequence are combined and encoded into an LED timing code sequence. Each LED timing code sequence is then matched and queried with the encoding and sensor signal mapping table in batches to obtain the LED sensor signal data reading results. The cross-validation includes: for data items with the same sensor parameter name after pairing, calculating the absolute deviation between the parameter value in the LED sensor signal data reading result and the parameter value in the structured screen data reading result; when the absolute deviation is within a preset allowable deviation range, calculating the weighted average of the parameter values ​​from the two sources according to a preset weight as the final reading; the preset weight is determined based on the historical reading deviation statistics of the LED reading channel and the tester screen reading channel, and the sum of the weights of the two channels is one; the final reading is equal to the product of the LED channel weight and the LED channel parameter value plus the product of the screen channel weight and the screen channel parameter value; when the absolute deviation exceeds the preset allowable deviation range, marking the workstation as an abnormal workstation and recording the deviation details. In addition, paired data with absolute deviation within the allowable range in each cross-validation are recorded in the historical calibration database, and the two-channel weights and parameter value correction coefficients corresponding to each coding mode in the coding and sensing signal mapping tables are periodically updated based on the accumulated data in the historical calibration database.

[0015] This invention provides an image recognition-based electronic atomizer sensor signal data reading system, comprising: The panoramic acquisition and target positioning module is used to acquire panoramic images of batch quality inspection stations. Through a dual-class target detection network, it simultaneously outputs the bounding box coordinates and station number labels of each LED indicator and the bounding box coordinates and instrument position labels of each tester screen, generating LED positioning index table and screen positioning index table. The halo compensation module is used to establish the spatial adjacency relationship between LED indicator lights and the tester screen based on the LED positioning index table and the screen positioning index table. It uses the point spread function to calculate the halo intensity value of each LED indicator light to each pixel position in the adjacent tester screen area and accumulates it pixel by pixel to generate a halo contribution matrix. It subtracts the halo contribution matrix pixel by pixel from the original image of the tester screen area to generate a halo-compensated screen sub-image. The screen layout parsing module is used to perform multi-scale edge detection and line detection on the halo-compensated screen sub-image to divide rectangular sub-regions. Based on the content type discrimination features of each rectangular sub-region, the rectangular sub-regions are classified into numerical reading regions, waveform display regions, status icon regions, or decorative regions. The screen data reading module is used to perform data reading for each category sub-region and generate structured screen data reading results. The LED brightness normalization and crosstalk compensation module is used to crop the independent sub-images of each LED indicator based on the LED positioning index table, and to perform adaptive brightness normalization and optical crosstalk compensation on the independent sub-images of each LED indicator. The LED timing code extraction module is used to extract the LED timing code sequence along the time axis of the video frame sequence and match it with the encoding and sensor signal mapping table to obtain the LED sensor signal data reading result. The dual-source cross-validation module is used to pair and cross-validate the LED sensor signal data reading results and structured screen data reading results corresponding to the same atomizer. It performs weighted fusion or anomaly marking based on the absolute deviation of the dual-source parameter values ​​and outputs a batch sensor signal data reading report.

[0016] This invention provides a method for reading sensor signal data from an electronic atomizer based on image recognition. It solves the technical problem of existing methods being unable to simultaneously complete LED indicator code recognition and tester screen data reading in batch quality inspection panoramic image acquisition scenarios, achieving the following technical effects: By estimating and compensating pixel-by-pixel the halo contribution based on the point spread function, it eliminates the reflective flare interference caused by the high-brightness active emission of LED indicators on the adjacent tester screen area, improving the input image quality for screen data reading; by using multi-scale Canny edge detection combined with Hough transform and connected component content type discrimination features for unsupervised analysis of the screen layout, the screen partitioning results are not dependent on specific products. The training samples, based on brand model or firmware version, can adapt to layout changes after instrument replacement or firmware upgrade. By performing dedicated brightness benchmark value normalization based on brightness histogram and ambient light component removal on the independent sub-images of each LED indicator, the interference of the tester screen backlight on LED brightness analysis is eliminated. Furthermore, optical crosstalk compensation between adjacent LED indicators reduces the impact of halo overlap between densely arranged LEDs on color and brightness discrimination. By performing dual-source cross-validation and weighted fusion on the LED encoding reading results and the tester screen reading results, the reading deviation of a single channel can be verified by the independent data of another channel, improving the reliability of batch sensor signal data reading results. Attached Figure Description

[0017] Figure 1 This is a flowchart of a batch quality inspection station sensor signal data reading method provided in an embodiment of the present invention; Figure 2 This is a schematic diagram illustrating the comparison of confidence levels for dual-class YOLO target detection provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the halo intensity contribution distribution of neighboring LEDs to pixels (1200, 454) provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the distribution of content type classification features in the sub-region of the INS02 screen provided in an embodiment of the present invention; Figure 5 This is a schematic diagram showing the correspondence between the pixel coordinates of the air flow rate scale in region E and the physical quantity provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of the timing distribution of the peak brightness of the W07LED provided in an embodiment of the present invention; Figure 7 This is a schematic diagram of the statistical analysis of adjacent brightness peak time intervals of W07 provided in an embodiment of the present invention; Figure 8 This is a schematic diagram of the comparison of dual-channel sensor parameter readings at the W07 workstation provided in an embodiment of the present invention. Detailed Implementation

[0018] At the batch quality inspection station at the end of the e-cigarette production line, dozens of atomizers are simultaneously placed on test fixtures and powered on. Each atomizer's LED indicator displays the sensor signal status using color and brightness encoding. Simultaneously, each atomizer is connected via cable to a nearby testing instrument displaying precise measurement data. Existing methods use a camera to capture video frame sequences of the LED indicator area of ​​a single e-cigarette, convert the color and brightness using HSV color space, and analyze the flashing frequency and color switching sequence along the time axis to generate LED timing codes. These codes are then matched with a sensor signal mapping table to obtain sensor signal data. For the testing instrument screen, a pre-trained U-Net segmentation network is used to divide the waveform display area and the numerical reading area, performing OCR recognition and waveform tracking respectively to read the instrument's measurement data.

[0019] However, in the scenario of panoramic image acquisition at the batch quality inspection station, the LED indicator lights of dozens of atomizers and the screens of multiple testers of different brands and models appear in the camera's field of view at the same time, which causes the following technical problems: First, the high brightness of the LED indicator light generates a large halo on the panoramic image, which creates reflective flares on the adjacent tester screen area, causing a large number of false edges and misjudgments of the tester screen area, resulting in the failure of the tester screen to read data.

[0020] Second, the backlight brightness change of the tester screen interferes with the brightness analysis of adjacent LED indicators. The uniform HSV brightness threshold cannot simultaneously adapt to the characteristics of two types of light sources: self-illuminating LED indicators and backlight-transmitting LCD screens, resulting in inaccurate determination of the brightness status of LED indicators.

[0021] Third, the screen layouts of the test instruments vary greatly among different instruments and may change after firmware upgrades. When the U-Net segmentation network encounters a new layout not included in the training set, the segmentation accuracy drops sharply, making it unable to quickly adapt to instrument replacement or firmware upgrade scenarios.

[0022] The aforementioned issues prevent existing methods from simultaneously completing the encoding and identification of all atomizer LED indicator lights and reading of tester screen data from a unified vision acquisition system, making it difficult to meet the batch sensor signal data reading requirements of high-speed production lines.

[0023] At least one embodiment of the present invention discloses a method for reading sensor signal data of an electronic atomizer based on image recognition, such as... Figure 1 As shown, it includes the following steps: According to an embodiment of this method, in terms of hardware environment, an industrial camera array is fixedly installed above the batch quality inspection station. The field of view of the industrial camera array covers all test fixture stations and adjacent test instruments. The intrinsic and extrinsic parameters of each camera have been pre-calibrated. The industrial camera array is connected to the back-end data processing server via an industrial Ethernet connection, and the video frame sequence is transmitted to the back-end data processing server in real time for processing. The correspondence between the station number of each atomizer on the test fixture and the instrument position label of the corresponding adjacent test instrument has been pre-configured and stored in the back-end data processing server.

[0024] Step 1: Acquire panoramic images and locate multiple target types; A panoramic video frame sequence of batch quality inspection stations is acquired by an industrial camera array. Each panoramic image is input into a dual-class YOLO detection network to perform multi-class target detection. At the same time, the bounding box coordinates and fixture station number label of each atomizer LED indicator, as well as the bounding box coordinates and instrument position label of each tester screen are output. These are then summarized to generate LED positioning index table and screen positioning index table.

[0025] It should be noted that the aforementioned dual-class YOLO detection network is a detection network obtained after training on a binary classification object detection task based on the YOLOv8 architecture. The input of the dual-class YOLO detection network is a panoramic image, and the output of the dual-class YOLO detection network is the bounding box coordinates, class labels, and confidence scores of the two types of targets: LED indicator lights and testing instrument screens. During the training phase, an annotated image dataset containing batch quality inspection workstation scenes is used. The annotations include the bounding box positions of LED indicator lights and testing instrument screens, as well as the corresponding workstation number labels or instrument position labels. The network parameters are trained under supervision using a detection loss function based on the intersection-over-union (IoU) ratio (which includes the sum of bounding box regression loss, classification loss, and confidence loss), and the network weights are updated using a stochastic gradient descent optimization algorithm.

[0026] It should be noted that the above LED positioning index table is a data structure with the workstation number as the index key and the LED bounding box coordinates as the value; the above screen positioning index table is a data structure with the instrument position label as the index key and the tester screen bounding box coordinates as the value.

[0027] Step 2: Estimate the halo contribution and compensate the screen area; Based on the LED positioning index table and the screen positioning index table, the Euclidean distance between the center coordinates of all LED indicator bounding boxes and the boundaries of all tester screen bounding boxes is calculated. Pairing relationships between LED indicators and tester screens with a distance less than a preset adjacency distance threshold are filtered out, and a spatial adjacency relationship table is generated.

[0028] For each tester screen bounding box region, all LED indicators adjacent to the tester screen bounding box region are traversed. Based on the bounding box center coordinates of each LED indicator and the peak brightness of the LED indicator sub-image, the halo intensity value of each LED indicator at each pixel position in the tester screen bounding box region is calculated using the point spread function. The halo intensity values ​​of all neighboring LED indicators within the same tester screen bounding box region are accumulated pixel by pixel to generate a halo contribution matrix. The halo contribution matrix is ​​then subtracted pixel by pixel from the original image of the tester screen bounding box region to generate the halo-compensated screen sub-image. The above processing is performed on all tester screen bounding box regions to generate a set of halo-compensated screen sub-images.

[0029] It should be noted that the point spread function described above is used to estimate the pixel count of a single LED indicator on the tester screen area. Halo intensity value The calculation method is as follows: in, For the first Peak brightness in the sub-image of each LED indicator light For pixels To the Center of each LED indicator frame The Euclidean distance (in pixels). This is the halo diffusion scale parameter (in pixels). It is a dimensionless decay exponent. and The dimensions of the LED indicator bounding box and the camera's optical parameters were pre-calibrated. and If two quantities have the same dimension, their ratio is a dimensionless quantity. and The dimensions are the same, and the dimensions of each term in the formula are consistent.

[0030] It should be noted that the above halo contribution matrix This is the cumulative result of the halo intensity values ​​of all adjacent LED indicator lights in the tester screen area: in, The number of LED indicator lights that are adjacent to the tester screen. This is the traversal index for the LED indicator lights. The screen sub-image after halo compensation. Calculate as follows: in, The original pixel values ​​of the tester screen area. The operation is used to truncate the compensated pixel values ​​to a non-negative range.

[0031] Step 3: Analyze the screen layout and categorize sub-regions; For each screen sub-image in the halo-compensated screen sub-image set, multi-scale Canny edge detection is performed, extracting three edge maps at low, medium, and high threshold levels respectively. These three edge maps are then logically ORed and merged to generate a screen content structure line map. Horizontal and vertical line segments are detected from the screen content structure line map based on Hough transform. Valid partition boundary lines are selected based on a preset ratio where the line segment length is greater than the corresponding side length of the screen sub-image, generating a set of partition boundary lines. Based on the intersection points and endpoint coordinates of the horizontal and vertical line segments in the partition boundary line set, the screen sub-image is divided into several rectangular sub-regions.

[0032] For each rectangular sub-region, content type discrimination features are extracted. Then, a rule-based decision tree is used to classify each rectangular sub-region into a numerical reading region, a waveform display region, a status icon region, or a decorative region, generating the partition classification results for each tester screen.

[0033] It should be noted that the above content type discrimination features include the following three categories: (1) Text density features: Binarization and connected component analysis are performed on the rectangular sub-region. The ratio of the total area of ​​the connected component to the area of ​​the rectangular sub-region and the average aspect ratio of the connected component are calculated. When the ratio is in the text density range and the average aspect ratio is close to the typical aspect ratio of the character, it is determined to be a text-dense region.

[0034] (2) Curve characteristics: The proportion of the area of ​​the elongated connected regions (connected regions whose ratio of major axis to minor axis is greater than the elongation ratio threshold) within the rectangular sub-region is counted; when this proportion is greater than the curve characteristic proportion threshold, it is determined to be a region with dense curves.

[0035] (3) Icon characteristics: The mean roundness and standard deviation of the area of ​​compact connected regions (connected regions whose ratio of major axis to minor axis is less than the compactness ratio threshold) within the rectangular sub-region are statistically analyzed; when the mean roundness is greater than the roundness threshold and the standard deviation of the area is less than the area dispersion threshold, it is determined to be an icon-dense region.

[0036] It should be noted that the rule-based decision tree described above performs classification according to the following priority: First, it determines whether the text density feature meets the condition of a dense text region. If it does, it is classified as a numerical reading region. Second, it determines whether the curve feature meets the condition of a dense curve region. If it does, it is classified as a waveform display region. Third, it determines whether the icon feature meets the condition of a dense icon region. If it does, it is classified as a status icon region. If none of the above conditions are met, it is classified as a decorative region.

[0037] Furthermore, in order to improve the adaptability of partition boundary line detection to the low-contrast tester screen content, before performing multi-scale Canny edge detection, CLAHE (contrast-limited adaptive histogram equalization) processing is performed on the halo-compensated screen sub-image to enhance the edge information of the weak contrast area in the screen sub-image, so that the subsequent multi-scale Canny edge detection can extract more complete partition boundary lines.

[0038] Step 4: Read the screen data and output it in a structured format; Connectivity analysis is performed on each rectangular sub-region marked as a numerical reading region in the partitioning and classification results. The height values ​​of the bounding rectangles of each connected region are statistically analyzed to obtain the character size distribution. Based on the median of the character size distribution, a matching OCR recognition model granularity is searched in a pre-defined OCR model granularity mapping table. The OCR recognition model with the matching granularity is used to perform character recognition on the numerical reading region to obtain the initial numerical recognition result. In the initial numerical recognition result, physical quantity unit symbols that are spatially adjacent to the numerical string are searched. The initial numerical recognition result is post-corrected based on the reasonable range of the numerical range corresponding to the physical quantity unit symbols. The numerical readings and their physical quantity labels for each numerical reading region are then output.

[0039] For each rectangular sub-region marked as a waveform display area in the partitioning and classification results, text connected components are searched within a pre-defined width strip area along the top, bottom, left, and right edges of the waveform display area. An OCR recognition model is used to perform character recognition on the searched text connected components to obtain scale values. Based on the correspondence between the pixel positions of each scale value in the edge strip area and the recognized scale values, a pixel-to-physical coordinate mapping matrix is ​​established through linear interpolation. Morphological skeleton extraction is performed on the waveform curves within the waveform display area to obtain a single-pixel-width curve skeleton. The pixel coordinate sequence of the curve skeleton is converted into a physical coordinate sequence using the pixel-to-physical coordinate mapping matrix, generating a numerical time-series data sequence of the sensor signal waveform.

[0040] Based on the spatial adjacency relationship of each rectangular sub-region within the screen sub-image, the numerical readings of the numerical reading area are semantically associated with the physical quantity labels of the numerical reading area and the numerical time-series data sequence of the waveform display area, generating structured screen data reading results corresponding to each instrument position label.

[0041] It should be noted that the above post-correction processing method is as follows: when the initial numerical recognition result exceeds the reasonable range of the numerical range corresponding to the physical quantity unit symbol, the candidate character replacement is performed on the character with the lowest confidence in the initial numerical recognition result (for example, swapping the number 0 with the letter O, and swapping the number 1 with the letter l), and the recognition result that falls into the reasonable range of the numerical range after replacement is selected as the corrected numerical reading.

[0042] It should be noted that the above pixel-physical coordinate mapping matrix is ​​established as follows: taking the horizontal direction as an example, the pixel ordinates of each tick mark in the left edge strip region are obtained. and its corresponding scale values ,in For the number of scale graduations, For the first The pixel ordinate of each tick mark. For the first Each scale character corresponds to a scale value, and adjacent scale pairs... and Linear interpolation is performed on the pixel positions between them to obtain the physical quantity value corresponding to the vertical coordinate of each pixel; the same process is applied to the vertical direction.

[0043] Furthermore, to improve the adaptability of waveform skeleton extraction to coarse-line waveforms, color clustering separation processing is performed on the waveform display area before morphological skeleton extraction. The color clustering separation processing uses the K-means clustering algorithm. The input of the K-means clustering algorithm is the RGB values ​​of each pixel in the waveform display area, and the output is the cluster label of each pixel. This separates the waveform curve pixels from the background pixels, allowing morphological skeleton extraction to be performed only on the waveform curve pixels, avoiding interference from background grid lines and annotation text. Here, K in the K-means clustering algorithm represents the preset number of clusters.

[0044] Step 5: Normalize the brightness of the LED sub-image and compensate for crosstalk; Based on the LED positioning index table, the bounding box regions of each LED indicator are cropped from the panoramic image, generating independent sub-images for each LED indicator. The following adaptive brightness normalization processing is then performed on each independent sub-image of the LED indicator: Step 501: Calculate the brightness histogram of the independent sub-image of the LED indicator. Calculate the average brightness value based on the set of pixels whose brightness values ​​in the brightness histogram are greater than the background brightness threshold, and use this as the exclusive brightness reference value for the LED indicator. .

[0045] Step 502: Obtain the tester screen boundary region adjacent to the LED indicator based on the spatial adjacency table, and calculate the average brightness value of the tester screen boundary region as the ambient light component. Subtract the ambient light component from the pixel brightness of each individual sub-image of the LED indicator. .

[0046] Step 503, based on the dedicated brightness reference value Calculate the normalization coefficient ,in The preset reference brightness value, and The ratio of two units with the same dimensions. The normalization factor is a dimensionless normalization coefficient, which is the product of the brightness of each pixel in the independent sub-image of the LED indicator after removing the ambient light component. Generate independent sub-images of LED indicator lights after brightness normalization.

[0047] For regions where there is spatial overlap between the bounding boxes of adjacent LED indicators, the halo diffusion model parameters of each LED indicator are estimated based on the hue distribution of each LED indicator in the HSV color space. The halo contribution of each LED indicator to the adjacent LED indicator region is calculated, and the halo contribution is subtracted pixel by pixel from the pixels of the adjacent LED indicator region. Optical crosstalk compensation is performed to generate independent sub-images of each LED indicator after crosstalk compensation.

[0048] It should be noted that the estimation method for the halo diffusion model parameters mentioned above is as follows: within the overlapping area of ​​two adjacent LED indicators, a sequence of pixel brightness values ​​is sampled along the line connecting the centers of the two LED indicators. The pixel brightness value sequence is then decomposed according to the hue weight of each LED indicator, and the diffusion scale parameter in the point diffusion function is fitted using the least squares method. and decay index The hue weighting is decomposed as follows: Let the hue value of a sampled pixel within the overlapping region be... The primary hue of the first LED indicator is The main hue of the second LED indicator is Then the pixel is assigned to the hue weight of the first LED indicator. and the hue weight assigned to the second LED indicator Calculate as follows: in, This represents the angular distance between two hue values ​​on the color wheel. The number index of the LED indicator is defined as follows: The unit is degrees. and The numerator and denominator have the same dimensions, both being angular distances, and their ratio is a dimensionless weight with consistent dimensions. Based on hue weight decomposition, the brightness contribution sequence along the center line is obtained for each LED indicator. Then, the diffusion scale parameter in the diffusion function of each LED indicator point is fitted using the least squares method. and decay index The point spread function has the same form as described in step 2.

[0049] Furthermore, to further improve the adaptability of brightness normalization to LED indicator lights with different luminous intensities, based on step 501, a bimodal detection is performed on the brightness histogram. When a bimodal distribution is detected (i.e., the LED indicator light is in a color switching transient state), the average brightness value corresponding to the higher brightness peak is taken as the exclusive brightness reference value. This avoids the intermediate brightness values ​​during color transitions affecting normalization accuracy.

[0050] Step 6: Extract the LED timing code and match it with the sensor signal; Parallel processing is performed on the independent sub-images of each LED indicator after crosstalk compensation along the time axis of the video frame sequence. For each LED indicator, the hue channel value and luminance channel value of the HSV color space are extracted in each frame image to generate the hue, time series and luminance, time series of the LED indicator.

[0051] Peak detection and zero-crossing detection are performed on the brightness and time series data to calculate the flicker frequency by measuring the time interval between adjacent brightness peaks. Step change detection is performed on the hue and time series data to extract the color switching times and the corresponding hue value sequences, generating a color switching sequence.

[0052] The flashing frequency and color switching sequence are combined and encoded into LED timing code sequences corresponding to each workstation number. Each LED timing code sequence is then matched and queried in batches with the corresponding atomizer model's encoding and sensor signal mapping table pre-stored in the backend data processing server to obtain the LED sensor signal data reading results of each workstation atomizer.

[0053] It should be noted that the above encoding and sensor signal mapping table is a data table pre-compiled according to the LED encoding rules corresponding to the atomizer model. Each record in the encoding and sensor signal mapping table contains an LED timing encoding mode and its corresponding sensor signal parameter name and parameter value. Batch matching query means using the LED timing encoding sequence as the query key to retrieve the corresponding record in the encoding and sensor signal mapping table.

[0054] It should be noted that the above-mentioned step change detection is performed as follows: the hue and the hue difference between adjacent frames in the time series are calculated. When the absolute value of the hue difference between adjacent frames exceeds the preset hue step threshold, it is determined that a color switch has occurred at that moment. The switching moment and the average hue value of the stable frame segment after the switch are recorded as the hue value corresponding to the switching moment, thereby extracting the color switching moment sequence and the corresponding hue value sequence.

[0055] Furthermore, in order to improve the robustness of flicker frequency calculation to frame rate fluctuations, cubic spline interpolation resampling is performed on the brightness and time series before peak detection, normalizing the unequal interval sampling times into an unequal interval time series, so that the subsequent peak interval calculation is not affected by camera frame rate jitter.

[0056] Furthermore, the time interval between adjacent brightness peaks in the aforementioned brightness time series... The calculation method is as follows: Let the first... The times when the brightness peaks appear in the video frame sequence are: , No. The time when the brightness peak occurs is ,in If the index is the sequence number of the brightness peak, then the time interval between adjacent peaks is... Units and times , Same, flicker frequency It is obtained by taking the reciprocal of the mean of multiple adjacent peak time intervals, i.e. ,in The arithmetic mean of the time intervals between all adjacent peaks. The unit is Hertz (Hz).

[0057] Step 7: Cross-validate the dual-source data and output the batch results; Based on the pre-stored correspondence between workstation number tags and instrument location tags in the backend data processing server, the LED sensor signal data reading results corresponding to the same atomizer are paired and associated with the structured screen data reading results.

[0058] For data items with the same sensor parameter name after pairing, calculate the absolute deviation between the parameter value in the LED sensor signal data reading result and the parameter value in the structured screen data reading result. When the absolute deviation is within the preset allowable deviation range, calculate the weighted average of the parameter values ​​from the two sources according to the preset weights as the final reading of the sensor parameter. When the absolute deviation exceeds the preset allowable deviation range, mark the station as an abnormal station and record the parameter values ​​from both sources and their deviation details.

[0059] Summarize the final readings of all workstations and the deviation details of abnormal workstations, and output a cross-validated batch sensor signal data reading report.

[0060] It should be noted that the above-mentioned preset weights are determined as follows: weights are assigned based on the historical reading deviation statistics of the LED reading channel and the tester screen reading channel, with the channel having a smaller historical reading deviation receiving a higher weight. LED Channel Weights and tester screen channel weights satisfy Final reading Calculate as follows: in, These are the parameter values ​​from the LED sensor signal data reading results. For the parameter values ​​in the structured screen data reading results, and All are dimensionless weights. and For the same sensing parameter with the same dimensions, all terms in the formula have the same dimensions.

[0061] Furthermore, to further utilize the cross-validation results to improve the reading accuracy of subsequent batches, paired data with absolute deviations within the allowable range in each cross-validation are recorded in the historical calibration database, and the LED channel weights are periodically updated based on the accumulated data in the historical calibration database. and tester screen channel weights The system also updates the parameter values ​​correction coefficients corresponding to each encoding mode in the encoding and sensor signal mapping table, so that the system gradually reduces the systematic deviation between the two channels as the running time increases.

[0062] This implementation method utilizes a dual-class YOLO detection network on a panoramic image to simultaneously output the localization results of two types of targets: LED indicator lights and the tester screen. This achieves parallel localization of multiple targets within a unified visual acquisition system, providing a spatial indexing basis for subsequent separate processing of the two types of targets.

[0063] This implementation estimates the halo contribution based on the point spread function of the spatial adjacency relationship between the LED indicator and the tester screen. It then calculates and subtracts the halo intensity component generated by each LED indicator on the adjacent tester screen area pixel by pixel, eliminating the reflective flare interference caused by the high-brightness active emission of the LED indicator on the tester screen area. Therefore, the halo-compensated screen sub-image no longer contains abrupt brightness gradient changes caused by the LED indicator halo. This eliminates false edges and misclassifications of partitions caused by halos in subsequent multi-scale Canny edge detection, improving the input image quality for tester screen data reading.

[0064] This implementation method performs unsupervised parsing of the tester screen layout by combining multi-scale Canny edge detection with Hough transform line detection and connected component content type discrimination features, replacing the U-Net segmentation network that relies on a specific training dataset. Because this parsing method directly partitions and classifies based on the geometric structure features and connected component statistical features of the tester screen content, and does not rely on tester screen layout samples from specific brands, models, or firmware versions for training, there is no issue of decreased segmentation accuracy when encountering new tester screen layouts not included in the training set. It can automatically adapt to layout changes after instrument replacement or firmware upgrades.

[0065] This implementation establishes an independent brightness reference system for each LED indicator by performing dedicated brightness reference value calculation and normalization processing based on the brightness histogram on the independent sub-image of each LED indicator, thus avoiding the difficulty of adapting a unified HSV brightness threshold to LED indicators with different luminous intensities. Simultaneously, by separating and removing the backlight brightness of the adjacent test instrument screen as an ambient light component from the brightness of the LED indicator area, the interference of the test instrument screen backlight on the LED indicator brightness analysis is eliminated. Furthermore, through optical crosstalk compensation between adjacent LED indicators, the mutual interference of color and brightness caused by halo overlap between densely arranged LED indicators is eliminated. Therefore, the color and brightness status discrimination of LED indicators is no longer affected by other light sources in a mixed light source scene, ensuring the accuracy of the input data extracted from the LED timing code.

[0066] This implementation uses dual-source cross-validation between the LED indicator code reading results and the tester screen reading results, utilizing the consistency relationship between the same sensor parameter values ​​acquired from two independent data channels for deviation detection and weighted fusion. Therefore, reading deviations that may occur in a single channel can be verified and corrected by the independent data from the other channel. This improves the reliability of batch sensor signal data reading results compared to methods relying on a single reading channel, enabling batch reading of atomizer sensor signal data from all workstations to be completed synchronously using a unified industrial camera array in high-speed production line scenarios.

[0067] The following is an example of an application of the present invention, such as... Figure 2-8 As shown, the implementation process is as follows: At a batch quality inspection station of an electronic atomizer manufacturer, there are 24 testing fixture stations arranged in 3 rows and 8 columns. Each row has 2 testing instruments (6 in total) on the side, and the instruments are of different brands. During a particular batch quality inspection task, an industrial camera array simultaneously captures panoramic video frame sequences from all 24 atomizers and 6 testing instruments, with a backend data processing server processing the image stream in real time. The atomizer with station number W07 (1st row, 7th column) is the primary tracking target. The corresponding testing instrument for this atomizer is labeled INS02, and the correspondence between the two has been pre-configured. The inspection objective is to read three sensor parameters from atomizer W07: heating temperature, atomization power, and airflow rate.

[0068] After the panoramic image is input into the dual-class YOLO detection network, the network simultaneously outputs 24 sets of LED bounding boxes and 6 sets of screen bounding boxes. Taking W07 and INS02 as examples, the detection results are shown in the table below. The LED positioning index table stores the bounding box coordinates with the workstation number as the key, and the screen positioning index table stores the corresponding coordinates with the instrument location label as the key, providing a spatial index basis for all subsequent steps.

[0069] Table 1. YOLO detection output results for both classes (W07 and INS02) Based on the LED positioning index table and the screen positioning index table, calculate the Euclidean distance between the center coordinates of the three LED indicators W06, W07, and W08 and the boundary of the INS02 screen. The center coordinates of the LED bounding box of W07 are (1140, 454)px. The left boundary of the INS02 screen bounding box is x=1180px. The horizontal distance from the center of W07 to the left boundary of INS02 is 1180 + 1140 = 40px, which is less than the preset adjacency distance threshold of 120px. Therefore, W07 (as well as W06 and W08) and INS02 form an adjacency pair, generating a spatial adjacency relationship table.

[0070] For the INS02 screen area, iterate through the adjacent LED indicators W06, W07, and W08, and calculate the halo intensity value of each LED for each pixel within the screen using the point spread function. Taking pixel (1200, 454) within the INS02 screen as an example, the Euclidean distance from pixel W07 to the center of W07 is: Given that W07 has a peak brightness of 218, a halo diffusion scale parameter of 45px, and an attenuation index of 2.4, then: The sum of the halo contributions of the three neighboring LEDs to the pixel is the sum of the contributions of each LED, as summarized in the table below.

[0071] Table 2. Contribution of three neighboring LEDs to the halo intensity of pixel (1200, 454) The total halo intensity G(1200,454) is calculated as 8.3 + 75.2 + 62.1 = 145.6. The original pixel value is 187, therefore the compensated pixel value is: The above pixel-by-pixel calculation is performed on all pixels of the INS02 screen area to generate a halo-compensated INS02 screen sub-image, and the brightness gradient abrupt change in the left edge area of ​​the screen (the side closer to the LED indicator) is effectively eliminated.

[0072] Step 3: Screen Image Enhancement and Region Segmentation After halo compensation, the INS02 screen sub-image was sequentially subjected to CLAHE enhancement and multi-scale Canny edge detection (low / medium / high threshold levels). The three edge maps were logically ORed and merged to generate a screen content structure line graph. Hough transform detected 7 valid horizontal line segments and 3 valid vertical line segments. The length of each line segment exceeded 40% of the preset ratio of the corresponding side length of the screen. Based on this, the INS02 screen sub-image (340×316px) was divided into 6 rectangular sub-regions.

[0073] Content type discrimination features were extracted for each rectangular sub-region, and the decision tree classification results are shown in the table below.

[0074] Table 3. Classification results of sub-regions of INS02 screen OCR recognition was performed on regions B, C, and D. The median height of the connected component characters was 18 pixels, and a standard granularity model was matched in the OCR model granularity mapping table. After recognizing the initial result, adjacent unit symbols were searched for post-correction. For example, region B initially recognized the value "2B3.7" with the unit symbol "℃", corresponding to a reasonable range of [0, 400]. Replacing the character "B" with the lowest confidence value with "8" resulted in "283.7", which falls within the reasonable range, and the correction was successful.

[0075] For region E (waveform display area), four scale characters were detected in the left edge strip area (width 12px). The correspondence between the pixel vertical coordinate and the scale value is shown in the table below.

[0076] Table 4 shows the correspondence between the left-side scale pixels and physical coordinates of region E. The pixel spacing between adjacent scale pairs (scale 1, scale 2) is 10px, corresponding to a physical quantity difference of 0.70L / min, with a linear interpolation ratio of 0.07L / min / px. K-means color clustering (K=3, corresponding to the waveform line, background, and grid lines respectively) is performed on the waveform skeleton pixel coordinate sequence within region E. Morphological skeleton extraction is performed only on the waveform line pixels, converting the ordinate of the skeleton pixels into physical values ​​of airflow rate to generate a numerical time-series data sequence. The final readings for each sub-region are summarized in the table below.

[0077] Table 5. Results of INS02 Structured Screen Data Reading Crop the LED sub-image (32×32px) of W07 from the panoramic image and perform adaptive brightness normalization.

[0078] Step 501: Calculate the brightness histogram. The average brightness of the set of pixels with brightness values ​​greater than the background brightness threshold (threshold set to 60) is 194.3. No bimodal distribution was detected, indicating that the current frame W07 is in a stable luminous state.

[0079] Step 502: Obtain the INS02 screen area adjacent to W07 from the spatial adjacency table, calculate the average brightness of the INS02 screen area as 31.6, and subtract 31.6 from the brightness of each pixel in the W07 sub-image.

[0080] Step 503: Set the preset reference brightness value to 200.0, and calculate the normalization coefficient: Multiply the brightness of each pixel in the W07 sub-image after removing ambient light by 1.029 to generate a brightness-normalized sub-image.

[0081] Furthermore, the bounding boxes of W07 and W08 have spatial overlap (overlap bandwidth approximately 18px). A luminance sequence of 16 pixels is sampled along the line connecting the centers of W07 and W08 within the overlapping area. The dominant hue of W07 is 127° (green), and the dominant hue of W08 is 22° (orange). For a sampled pixel within the overlapping area with a hue value of 89°, the hue weight is calculated as follows: Based on the decomposed brightness contribution sequence, the least squares method was used to fit the diffusion scale parameter of W07 as 43.8px and the attenuation index as 2.3. Pixel-by-pixel crosstalk compensation was performed on the W08 region to generate the crosstalk-compensated independent sub-image of W07.

[0082] Hue, time series, and luminance, time series were extracted from the W07 crosstalk compensated sub-image along the video frame sequence time axis. After performing cubic spline interpolation on the luminance and time series and resampling to the equally spaced time axis, five luminance peaks were detected, and the time intervals between adjacent peaks are shown in the table below.

[0083] Table 6. Statistics on peak brightness time intervals for W07 The average time interval is (166.4 + 166.7 + 166.5 + 166.6) / 4 = 166.55 ms, and the flicker frequency is: A hue step change was detected at frame 124 (the hue difference jumped from 127° to 22°, exceeding the threshold of 30°), and the color switching sequence was "green → orange". The flashing frequency of 6.0Hz was combined with the color switching sequence "green → orange" to form the LED timing code for W07. A match query was performed in the code and sensor signal mapping table, and the following records were found: heating temperature 282.5℃, atomization power 18.1W, and air flow rate 2.60L / min.

[0084] Based on the pre-stored workstation and instrument correspondence in the backend data processing server, the LED sensor signal readings from W07 are paired and associated with the structured screen readings from INS02. The absolute deviation of the three common sensing parameters is calculated, and a weighted final reading is calculated according to preset weights (0.62 for the LED channel due to its smaller historical deviation; 0.38 for the screen channel). Taking heating temperature as an example: The cross-validation results of the three parameters are shown in the table below. All three absolute deviations are within the allowable deviation range (temperature ±2℃, power ±0.5W, air flow ±0.1L / min). Station W07 is marked as normal, and the paired data is recorded to the historical calibration database.

[0085] Table 7. Dual-source cross-validation and final readings at workstation W07 The entire data stream starts with a panoramic image. Step 1 uses dual-class YOLO detection to output the LED bounding box coordinates of W07 and the screen bounding box coordinates of INS02, constructing a spatial index. Step 2 uses these two sets of coordinates to calculate spatial adjacency relationships, estimating the halo contribution pixel-by-pixel based on the peak brightness and position parameters of neighboring LEDs such as W07, and then passes the compensated INS02 screen sub-image to Step 3. Step 3 parses six rectangular sub-regions and their classifications based on the geometric structure of the screen sub-image, and passes the partitioning results to Step 4. Step 4 performs OCR correction on the numerical reading area and establishes pixel and object mapping for the waveform area. Step 5: Using the LED bounding box from Step 1, crop the W07 sub-image and obtain the ambient light component using the spatial adjacency relationship established in Step 2, completing brightness normalization and crosstalk compensation; Step 6: Extract the timing code along the time axis from the compensated sub-image output in Step 5 and match it with the mapping table to output LED channel sensing data; Step 7: Merge the two independent readings from Steps 4 and 6 according to preset weights, and finally output the cross-validated W07 station sensing parameter report. The data transmission relationship between each step strictly corresponds, ensuring the traceability and consistency of the final reading.

[0086] The embodiments of the present invention have been described above. However, the embodiments are not limited to the specific implementation methods described above. The specific implementation methods described above are merely illustrative and not restrictive. Those skilled in the art can make more equivalent embodiments under the guidance of the present embodiments, and all of them are within the protection scope of the present embodiments.

Claims

1. A method for reading sensor signal data of an electronic atomizer based on image recognition, characterized in that, Includes the following steps: Collect panoramic images of batch quality inspection stations, and simultaneously output the bounding box coordinates and station number labels of each LED indicator and the bounding box coordinates and instrument position labels of each tester screen through a dual-class target detection network, generating an LED positioning index table and a screen positioning index table. Based on the LED positioning index table and the screen positioning index table, establish the spatial adjacency relationship between the LED indicator and the tester screen. Use the point spread function to calculate the halo intensity value of each LED indicator for each pixel position in the adjacent tester screen area. Accumulate the halo intensity values ​​of all adjacent LED indicators pixel by pixel to generate the halo contribution matrix. Subtract the halo contribution matrix pixel by pixel from the original image of the tester screen area to generate the halo-compensated screen sub-image. Multi-scale edge detection and line detection are performed on the halo-compensated screen sub-image to divide rectangular sub-regions. Based on the content type discrimination features of each rectangular sub-region, the rectangular sub-regions are classified into numerical reading regions, waveform display regions, status icon regions, or decorative regions. Data reading is performed on each classified sub-region to generate structured screen data reading results. Based on the LED positioning index table, each LED indicator independent sub-image is cropped, adaptive brightness normalization and optical crosstalk compensation are performed on each LED indicator independent sub-image, the LED timing encoding sequence is extracted along the time axis of the video frame sequence, and the LED sensing signal data reading result is obtained by matching with the encoding and sensing signal mapping table. The LED sensor signal data reading results corresponding to the same atomizer are paired, correlated, and cross-validated with the structured screen data reading results. Weighted fusion or anomaly marking is performed based on the absolute deviation of the dual-source parameter values, and a batch sensor signal data reading report is output.

2. The method for reading sensor signal data of an electronic atomizer based on image recognition according to claim 1, characterized in that, The calculation of the halo intensity value of each LED indicator at each pixel position in the adjacent tester screen area using the point spread function includes: Calculate the Euclidean distance between the center coordinates of all LED indicator bounding boxes and the boundaries of all tester screen bounding boxes, filter the pairing relationships of LED indicators and tester screens with a distance less than a preset adjacency distance threshold, and generate a spatial adjacency table; The point spread function is calculated as follows: the halo intensity value of a single LED indicator at a certain pixel position in the tester screen area is equal to the peak brightness of the LED indicator sub-image divided by the sum of the attenuation exponent obtained by adding the ratio of the Euclidean distance from the pixel position to the center of the LED indicator bounding box to the halo spread scale parameter, wherein the halo spread scale parameter and the attenuation exponent are obtained in advance based on the LED indicator bounding box size and camera optical parameters. The halo contribution matrix is ​​generated by accumulating the halo intensity values ​​of all adjacent LED indicators within the same tester screen area pixel by pixel. The corresponding values ​​of the halo contribution matrix are then subtracted from the original pixel values ​​of the tester screen area pixel by pixel, and the compensated pixel values ​​are truncated to a non-negative range.

3. The method for reading sensor signal data of an electronic atomizer based on image recognition according to claim 1, characterized in that, The process involves performing multi-scale edge detection and line detection on the halo-compensated screen sub-image to divide it into rectangular sub-regions, and classifying the rectangular sub-regions based on the content type discrimination features of each rectangular sub-region, including: Canny edge detection is performed on the halo-compensated screen sub-image at low, medium, and high threshold levels to extract three edge maps. The three edge maps are then logically ORed and merged to generate a screen content structure line diagram. Based on Hough transform, horizontal and vertical line segments are detected from the screen content structure line graph. Valid partition boundary lines are selected according to the condition that the line segment length is greater than the corresponding side length of the screen sub-image by a preset ratio. Based on the intersection and endpoint coordinates of the partition boundary lines, the screen sub-image is divided into several rectangular sub-regions. Content type discrimination features are extracted for each rectangular sub-region. These features include text density features, curve features, and icon features. The text density features are obtained by performing binarization and connected component analysis on the rectangular sub-region, calculating the ratio of the total area of ​​the connected components to the area of ​​the rectangular sub-region, as well as the average aspect ratio of the connected components. The curve features are obtained by statistically analyzing the proportion of the area of ​​slender connected components within the rectangular sub-region whose ratio of the major axis to the minor axis is greater than the slenderness ratio threshold. The icon features are obtained by statistically analyzing the mean roundness and standard deviation of the area of ​​compact connected components within the rectangular sub-region whose ratio of the major axis to the minor axis is less than the compactness ratio threshold. The classification is performed using a rule-based decision tree with the following priority: First, it is determined whether the text density feature meets the condition of a dense text region. If it does, it is classified as a numerical reading region. Second, it is determined whether the curve feature meets the condition of a dense curve region. If it does, it is classified as a waveform display region. Third, it is determined whether the icon feature meets the condition of a dense icon region. If it does, it is classified as a status icon region. If none of the above conditions are met, it is classified as a decorative region.

4. The method for reading sensor signal data of an electronic atomizer based on image recognition according to claim 3, characterized in that, Before performing multi-scale Canny edge detection, contrast-limited adaptive histogram equalization is also performed on the halo-compensated screen sub-image to enhance the edge information of weak contrast areas in the screen sub-image.

5. The method for reading sensor signal data of an electronic atomizer based on image recognition according to claim 1, characterized in that, The step of performing data reading for each category sub-region includes: Connectivity analysis is performed on the numerical reading region to obtain the character size statistical distribution by statistically analyzing the height values ​​of the bounding rectangles of each connected region. Based on the median of the character size statistical distribution, a matching OCR recognition model granularity is searched in a preset OCR model granularity mapping table. The OCR recognition model with the matching granularity is used to perform character recognition on the numerical reading region to obtain the initial numerical recognition result. The physical quantity unit symbol adjacent to the numerical string space is searched in the initial numerical recognition result. The initial numerical recognition result is post-corrected based on the reasonable range of the numerical range corresponding to the physical quantity unit symbol, and the numerical reading and physical quantity label are output. The post-correction is as follows: when the initial numerical recognition result exceeds the reasonable range of the numerical range, the character with the lowest confidence in the initial numerical recognition result is replaced with a candidate character, and the recognition result that falls into the reasonable range of the numerical range after replacement is selected as the corrected numerical reading. The text connected components are searched within a strip of preset width at the top, bottom, left, and right edges of the waveform display area. The OCR recognition model is used to perform character recognition on the text connected components to obtain scale values. Based on the correspondence between the pixel positions of each scale value in the edge strip area and the scale values, a pixel-physical coordinate mapping matrix is ​​established through linear interpolation. Morphological skeleton extraction is performed on the waveform curve in the waveform display area to obtain a curve skeleton with a single pixel width. The pixel coordinate sequence of the curve skeleton is converted into a physical coordinate sequence using the pixel-physical coordinate mapping matrix to generate a numerical time series data sequence. Based on the spatial adjacency of each rectangular sub-region within the screen sub-image, the numerical readings are semantically associated with physical quantity labels and numerical time-series data sequences to generate structured screen data reading results.

6. The method for reading sensor signal data of an electronic atomizer based on image recognition according to claim 5, characterized in that, Before performing morphological skeleton extraction on the waveform display area, color clustering separation processing is also performed on the waveform display area. The color clustering separation processing adopts the K-means clustering algorithm. The input of the K-means clustering algorithm is the RGB value of each pixel in the waveform display area, and the output of the K-means clustering algorithm is the cluster label of each pixel. After separating the waveform curve pixels from the background pixels, morphological skeleton extraction is performed only on the waveform curve pixels.

7. The method for reading sensor signal data of an electronic atomizer based on image recognition according to claim 1, characterized in that, The adaptive brightness normalization and optical crosstalk compensation performed on the independent sub-images of each LED indicator includes: Calculate the brightness histogram of the independent sub-image of the LED indicator, and calculate the average brightness value as the exclusive brightness benchmark value based on the set of pixels whose brightness value is greater than the background brightness threshold in the brightness histogram; The tester screen boundary region adjacent to the LED indicator is obtained based on spatial adjacency relationship. The average brightness value of the tester screen boundary region is calculated as the ambient light component. The ambient light component is subtracted from the brightness of each pixel in the independent sub-image of the LED indicator. The normalization coefficient is calculated based on the exclusive brightness reference value. The normalization coefficient is equal to the ratio of the preset reference brightness value to the exclusive brightness reference value. The brightness of each pixel after removing the ambient light component is multiplied by the normalization coefficient to generate an independent sub-image of the LED indicator after brightness normalization. For regions where there is spatial overlap between the bounding boxes of adjacent LED indicator lights, the pixel brightness value sequence is sampled along the line connecting the centers of the two LED indicator lights within the overlapping region. The pixel brightness value sequence is decomposed according to the hue weight of each LED indicator light. The diffusion scale parameter and attenuation index in the point spread function are fitted using the least squares method. The halo contribution of each LED indicator light to the adjacent LED indicator light region is calculated, and the halo contribution is subtracted pixel by pixel from the pixels of the adjacent LED indicator light region. The hue weight is decomposed as follows: the hue weight of a sampled pixel in the overlapping area belonging to the first LED indicator is equal to the hue ring angle distance between the hue value of the sampled pixel and the main hue of the second LED indicator divided by the sum of the hue ring angle distances between the hue value of the sampled pixel and the main hues of the two LED indicators; the hue weight belonging to the second LED indicator is equal to the hue ring angle distance between the hue value of the sampled pixel and the main hue of the first LED indicator divided by the sum of the above angle distances.

8. The method for reading sensor signal data of an electronic atomizer based on image recognition according to claim 7, characterized in that, When calculating the exclusive brightness reference value, a bimodal distribution is also performed on the brightness histogram. When a bimodal distribution is detected, the average brightness value corresponding to the higher brightness peak is taken as the exclusive brightness reference value.

9. The method for reading sensor signal data of an electronic atomizer based on image recognition according to claim 1, characterized in that, The extraction of the LED timing-coded sequence along the time axis of the video frame sequence includes: For each LED indicator, extract the hue channel value and luminance channel value of the HSV color space in each frame of the image to generate hue, time series and luminance, time series; After performing cubic spline interpolation resampling on the brightness and time series to normalize the unequally spaced sampling times into an unequally spaced time series, peak detection is performed, the time interval between adjacent brightness peaks is calculated, and the flicker frequency is obtained by taking the reciprocal of the arithmetic mean of multiple adjacent peak time intervals. A step change detection is performed on the hue and time series. The step change detection is to calculate the hue difference between adjacent frames in the hue and time series. When the absolute value of the hue difference between adjacent frames exceeds the preset hue step threshold, it is determined that a color switch has occurred. The switching time and the average hue value of the stable frame segment after the switch are recorded to generate a color switch sequence. The flashing frequency and color switching sequence are combined and encoded into an LED timing code sequence. Each LED timing code sequence is then matched and queried with the encoding and sensor signal mapping table in batches to obtain the LED sensor signal data reading results. The cross-validation includes: for data items with the same sensor parameter name after pairing, calculating the absolute deviation between the parameter value in the LED sensor signal data reading result and the parameter value in the structured screen data reading result; when the absolute deviation is within a preset allowable deviation range, calculating the weighted average of the parameter values ​​from the two sources according to a preset weight as the final reading; the preset weight is determined based on the historical reading deviation statistics of the LED reading channel and the tester screen reading channel, and the sum of the weights of the two channels is one; the final reading is equal to the product of the LED channel weight and the LED channel parameter value plus the product of the screen channel weight and the screen channel parameter value; when the absolute deviation exceeds the preset allowable deviation range, marking the workstation as an abnormal workstation and recording the deviation details. In addition, paired data with absolute deviation within the allowable range in each cross-validation are recorded in the historical calibration database, and the two-channel weights and parameter value correction coefficients corresponding to each coding mode in the coding and sensing signal mapping tables are periodically updated based on the accumulated data in the historical calibration database.

10. A system for reading sensor signal data of an electronic atomizer based on image recognition, used to execute the method for reading sensor signal data of an electronic atomizer based on image recognition as described in any one of claims 1 to 9, characterized in that, include: The panoramic acquisition and target positioning module is used to acquire panoramic images of batch quality inspection stations. Through a dual-class target detection network, it simultaneously outputs the bounding box coordinates and station number labels of each LED indicator and the bounding box coordinates and instrument position labels of each tester screen, generating LED positioning index table and screen positioning index table. The halo compensation module is used to establish the spatial adjacency relationship between LED indicator lights and the tester screen based on the LED positioning index table and the screen positioning index table. It uses the point spread function to calculate the halo intensity value of each LED indicator light to each pixel position in the adjacent tester screen area and accumulates it pixel by pixel to generate a halo contribution matrix. It subtracts the halo contribution matrix pixel by pixel from the original image of the tester screen area to generate a halo-compensated screen sub-image. The screen layout parsing module is used to perform multi-scale edge detection and line detection on the halo-compensated screen sub-image to divide rectangular sub-regions. Based on the content type discrimination features of each rectangular sub-region, the rectangular sub-regions are classified into numerical reading regions, waveform display regions, status icon regions, or decorative regions. The screen data reading module is used to perform data reading for each category sub-region and generate structured screen data reading results. The LED brightness normalization and crosstalk compensation module is used to crop the independent sub-images of each LED indicator based on the LED positioning index table, and to perform adaptive brightness normalization and optical crosstalk compensation on the independent sub-images of each LED indicator. The LED timing code extraction module is used to extract the LED timing code sequence along the time axis of the video frame sequence and match it with the encoding and sensor signal mapping table to obtain the LED sensor signal data reading result. The dual-source cross-validation module is used to pair and cross-validate the LED sensor signal data reading results and structured screen data reading results corresponding to the same atomizer. It performs weighted fusion or anomaly marking based on the absolute deviation of the dual-source parameter values ​​and outputs a batch sensor signal data reading report.

Images