Infrared and visible light touch multi-source fusion robust recognition method

Abstract

The application discloses an infrared and visible light touch multi-source fusion robust recognition method, relates to the technical field of optical touch recognition, and comprises the following steps: collecting infrared data in a short time window to extract an environmental infrared interference fingerprint, and combining visible light fingertip information to generate infrared reliability and visible light reliability; selecting an emission code sequence and emission parameters according to the fingerprint, emitting in a verification time window, and obtaining a correlation demodulation quality index by matching the correlation demodulation; triggering a candidate touch area by visible light, emitting a one-time verification code sequence, and forming a touch verification record by the relevant matching of the candidate touch area at the receiving end; performing quality index, verification record and reliability gate control fusion decision, outputting a touch event through a hysteresis state machine, and closed-loop adjusting emission and receiving parameters; the method reduces false touch and missed touch under strong light saturation, lighting flicker and external infrared interference, enhances the anti-fraud touch resistance, and controls the delay.

Description

Technical Field

[0001] This invention relates to the field of optical touch recognition technology, specifically a robust recognition method that integrates infrared and visible light touch sources. Background Technology

[0002] In public spaces such as smart government, transportation hubs, hospitals, and retail, interactive devices like self-service ticketing and payment terminals, ticket gates, outdoor information kiosks, interactive exhibits in museums, in-vehicle central control systems, and large-screen displays for conferences and educational settings all utilize graphical user interfaces (without mechanical buttons) for multi-touch control. To accommodate large sizes, thick glass covers, non-standard bezels, and projection interaction surfaces, infrared optical touch control (infrared frames, diffuse illumination, total internal reflection contact imaging, etc.) and infrared touch detection combined with visible light imaging are employed in combination. Touch points or contact events are extracted using methods such as threshold segmentation, morphology, connected components, trajectory correlation, background updates, inter-frame difference, or fixed modulation detection. In engineering applications, narrowband filtering, automatic exposure, transmit power adjustment, or light-shielding structures are used as auxiliary technologies.

[0003] However, in actual deployments, terminals are often placed near glass curtain walls, semi-outdoor corridors, subway entrances, and areas under direct sunlight. Ambient infrared radiation and strong visible light cause infrared receiver saturation, dynamic range compression, enhanced scattering and reflection, and background drift. The flickering of indoor lighting fixtures generates periodic noise that changes over time. Explosion-proof films, films, and anti-reflective coatings covering glass surfaces alter the reflection spectrum and scattering distribution, causing non-uniform high-brightness areas of ambient infrared radiation in the image, which move rapidly with changes in the angle of incidence and the position of the person. Because the amplitude of contact signals is limited and often superimposed on surface reflections, the threshold and background model are prone to drift and misjudgment after interference enhancement. When using inter-frame differential, rapid finger movement, rolling shutter scanning, synchronization errors, or image jitter can also cause differential residue and false contact points. External infrared sources such as security supplementary lighting, infrared emission from adjacent terminals, and remote controls are superimposed on the receiving channel, and the superposition of contact evidence and background noise causes the contact point judgment boundary to change over time. If the above instabilities occur in high-risk interfaces such as payment confirmation, gate access, and industrial control, they can lead to accidental triggering, missed touches and no response, touch point jitter, interruption of business processes, instruction errors, increased operation and maintenance costs, and limit the deployable range in high-light environments. At the same time, the rapid and continuous operation by multiple people during high-traffic periods will also exacerbate the above instabilities.

[0004] Therefore, the technical problem faced by existing technologies is:

[0005] Under deployment conditions where strong ambient light and external infrared interference coexist and change over time, infrared touch control and its recognition link combined with visible light have difficulty in stably distinguishing between real touch events and pseudo-touch events caused by factors such as saturation, reflection, flicker, synchronization errors and timing mismatches, which in turn leads to accidental touches, missed touches and discontinuous contact timing. Summary of the Invention

[0006] (a) Technical problems to be solved

[0007] To address the shortcomings of existing technologies, this invention provides a robust multi-source fusion identification method for infrared and visible light touch control. This method involves selecting a transmission code sequence and transmission parameters based on a fingerprint, transmitting and matching relevant demodulation within a verification time window to obtain relevant demodulation quality indicators; triggering candidate touch areas with visible light and transmitting a one-time verification code sequence; the receiver only matches the candidate touch areas to form a touch verification record; gating and fusing the quality indicators, verification records, and reliability for decision-making; outputting the touch event via a hysteresis state machine; and adjusting the transmission and reception parameters in a closed loop. This method reduces false and missed touches under conditions of strong light saturation, lighting flicker, and external infrared interference, thus solving the technical problems described in the background art.

[0008] (II) Technical Solution

[0009] To achieve the above objectives, the present invention provides the following technical solution:

[0010] A robust multi-source fusion identification method for infrared and visible light touch control includes: collecting infrared received data in short time windows during device operation, calculating environmental infrared interference fingerprints, and extracting background brightness, saturation ratio, short-term drift and flicker components; simultaneously collecting visible light hand and fingertip information, summarizing and recording infrared reliability and visible light reliability;

[0011] Based on the environmental infrared interference fingerprint and infrared reliability, the transmission code sequence and transmission parameters are selected from the preset codebook, and the transmission code sequence is transmitted within the verification time window; the receiving end samples in the interaction area and performs matching correlation demodulation, and outputs the correlation demodulation quality index;

[0012] The visible light side generates candidate touch areas and provides candidate touch confidence levels based on visible light reliability triggering, and transmits a one-time verification code sequence; the receiving end only performs matching-related demodulation within the candidate touch areas and matches the one-time verification code sequence to form a touch verification record;

[0013] Based on the gating fusion rules, the relevant demodulation quality indicators, candidate touch confidence, infrared reliability and touch verification records are judged and the touch event is output through the hysteresis state machine. The transmission parameters and the receiving exposure and gain are updated according to the environmental infrared interference fingerprint.

[0014] Furthermore, within each short time window, statistics are performed on the full screen of the interactive area and the suspected touch neighborhood of the visible light prompts to generate an environmental infrared interference fingerprint. The infrared reliability and visible light reliability are represented by continuous values ​​with level labels, and written into the reliability record along with the timestamp as the input basis for steps two and four.

[0015] Furthermore, the environmental infrared interference fingerprint includes background brightness, saturation ratio, short-time drift and flicker components. The flicker component is synthesized by projecting the statistical sequence of background brightness within a short time window onto the sine basis function and cosine basis function respectively, and the dominant frequency and harmonic energy are extracted, and the dominant frequency position is written into the reliability record.

[0016] Furthermore, in the preset codebook, the code spectrum collision amount is calculated for each candidate transmission code sequence according to the main frequency of the scintillation component, and a candidate code sequence set is formed by combining the sidelobe cost. Then, the transmission code sequence and transmission parameters are determined from the candidate code sequence set based on the infrared reliability and written into the encoding and demodulation record, and the index of the transmission code sequence is saved.

[0017] Furthermore, in the matching correlation demodulation, the interaction region is sampled within the verification time window to form a received sampling sequence, the transmit code sequence is zero-filled and shifted, and the correlation response sequence is calculated within a limited shift range. Based on the peak candidate window and the sidelobe window, the correlation peak value, peak-side ratio, peak position consistency mark and peak shape stability mark are obtained as correlation demodulation quality indicators.

[0018] Furthermore, the candidate touch area is generated centered on the fingertip and mapped onto the infrared receiving image. On the visible light side, when the condition of continuous frames where the fingertip enters the interaction area and the speed decreases and remains stable is met, a verification time window for a one-time verification code sequence is triggered, and repeated triggering is suppressed within the same verification time window.

[0019] Furthermore, the one-time verification code sequence is generated by bitwise XOR, cyclic shift and byte permutation of key and timestamp to generate index. The corresponding transmission code sequence is selected from the candidate code sequence set and used for this verification. The receiving end determines the matching template according to the index and compares it with the relevant response sequence, and writes the index and timestamp into the touch verification record.

[0020] Furthermore, the relevant response sequence is calculated only within the candidate touch area. When the relevant peak position falls into the peak candidate window and the peak-side ratio reaches the threshold associated with infrared reliability and scintillation component, and satisfies the geometric proximity relationship with the fingertip, a valid verification conclusion of the touch verification record is generated.

[0021] Furthermore, the relevant demodulation quality indicators are mapped to infrared verification confidence, and the candidate touch confidence is combined with visible light reliability to map to visible light touch confidence. Based on the infrared reliability, the gating fusion rules are selected and the touch events are output. When the infrared reliability decreases, the system enters a conservative mode and the confirmation conditions are improved.

[0022] Furthermore, the hysteresis state machine transitions to four states: hovering, candidate, contact, and release. Entering and exiting contact are subject to different continuous verification time window counting conditions. A status identifier is maintained for the same candidate touch area. When multiple candidate touch areas overlap, the attribution of the touch event is determined based on the infrared verification confidence level and peak position consistency mark.

[0023] Furthermore, the transmission parameters, reception exposure, and gain are updated based on the environmental infrared interference fingerprint and touch verification records. The updated values ​​are then projected onto a preset set of levels and written into the update buffer. This process takes effect at the boundaries of adjacent verification time windows and keeps the transmission code sequence and transmission parameters unchanged within the same verification time window.

[0024] Furthermore, the receiver is an infrared camera with external trigger exposure. The controller outputs an external trigger signal to the infrared camera at the chip rate, so that it performs short exposure sampling in each chip cycle to form a receiving sampling sequence, and reads it out in the pixel set corresponding to the candidate touch area, and makes the verification time window correspond to the code length and chip rate in the whole cycle.

[0025] Furthermore, before statistics are performed, an avoidance mask is generated based on the hand and fingertip information on the visible light side and applied to the infrared received data. The median of the neighborhood is replaced by a constant outlier table, and an edge shielding band is set to remove stray reflection pixels. After performing morphological closing operations on the boundary of the avoidance mask, background statistical channels and candidate neighborhood statistical channels are formed respectively.

[0026] (III) Beneficial Effects

[0027] This invention provides a robust multi-source fusion recognition method for infrared and visible light touch, which has the following advantages:

[0028] Ambient infrared interference fingerprint is extracted in a short time to form infrared and visible light reliability. As the selection of subsequent transmission code sequences and fusion decisions are affected by strong light, flicker, and drift, misjudgments caused by fixed threshold drift are adaptively addressed. Based on the ambient infrared interference fingerprint, the transmission code sequence and transmission parameters are selected, and transmission and matching demodulation are performed within the verification time window to generate correlation peaks and correlation demodulation quality indicators. This ensures that the active infrared component completes background suppression within the same time window, avoiding motion residuals and temporal artifacts associated with frame difference methods.

[0029] Visible light triggers candidate touch areas and emits a one-time verification code sequence. The receiving end only matches and forms touch verification records in the candidate touch areas, so that touch confirmation meets the fingertip geometric constraints and the same code evidence, suppressing false touch caused by external supplementary light, environmental fluctuations and malicious infrared injection.

[0030] Relevant demodulation quality indicators, touch verification records, and reliability are incorporated into the gating fusion rules. Touch events are then output by the hysteresis state machine, ensuring stable transitions between contact and release across multiple frames, reducing touch jitter and transient interference triggers, and ensuring consistent input across interfaces such as payment confirmation and gate access. Transmission parameters and receiver exposure and gain are updated in a closed loop at the time window boundaries, linking candidate touch areas and confirmation thresholds. This makes the system usable and controls interaction latency in scenarios with simultaneous semi-outdoor direct sunlight, glass reflection, and lighting flicker, reducing the frequency of on-site parameter tuning.

[0031] By coordinating environmental infrared interference fingerprint-driven encoded transmission, one-time verification code verification within the candidate touch area, and gated fusion hysteresis output, the solution integrates environmental assessment, evidence generation, and event output into the same recording link, enabling multi-source information to be aligned and traceable within the same verification time window, thereby improving system-level robustness and anti-fraud capabilities. Attached Figure Description

[0032] Figure 1 This is a schematic diagram of the robust recognition method for infrared and visible light touch multi-source fusion according to the present invention. Detailed Implementation

[0033] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0034] Please see Figure 1 This invention provides a robust multi-source fusion recognition method for infrared and visible light touch control, including:

[0035] Step 1: Without affecting device interaction, combine the infrared reception sequence and visible light observation results in the sampling time window to form an environmental infrared interference fingerprint, and output infrared reliability and visible light reliability, in order to provide reusable environmental and quality markers for subsequent steps.

[0036] In scenarios involving strong ambient light, external infrared illumination, enhanced reflection from glass overlays, and flickering indoor light sources, infrared touch reception can exhibit changes such as expanded saturation areas, increased background brightness, and background drift. These changes, combined with the localized response caused by touch, can lead to instability in the evidence relied upon for subsequent touch confirmation. While visible light can acquire geometric information about the hand and fingertips, it is also susceptible to overexposure, underexposure, and motion blur, resulting in unstable localization and persistence of candidate touch areas.

[0037] Therefore, step one requires explicit labeling of environmental conditions and observation quality: on the one hand, using environmental infrared interference fingerprints to characterize the intensity, time, and period of the infrared background; on the other hand, using infrared reliability and visible light reliability to characterize the availability of the two types of observations within the current time range, avoiding subsequent steps from directly performing touch confirmation under unknown environmental conditions. First, the infrared received image and visible light observation results are aligned with timestamps, allowing subsequent environmental descriptions to be based on the same time period. Then, candidate touch areas are obtained using hand and fingertip detection results from the visible light side, and these candidate touch areas are mapped onto the infrared received image according to coordinates to form an avoidance mask, ensuring that the environmental fingerprint avoids the hand-covered area as much as possible, reducing the possibility of fingerprint contamination by the touch itself.

[0038] During device operation, the infrared receiver continuously outputs infrared received images and records the start and end times of each sampling time window, along with the exposure duration and analog gain of the infrared receiver. Subsequently, the exposure duration and analog gain remain unchanged within the sampling time window to avoid pseudo-drift introduced by automatic exposure jumps within the same sampling time window.

[0039] On the visible light side, the hand contour and fingertip position are output within the same sampling time window, and the fingertip position is expanded into a candidate touch area. After the candidate touch area is mapped onto the infrared receiving image via camera extrinsic parameters, an avoidance mask is formed, which is used to mark the set of pixels that need to be removed from the environmental statistics. For extrinsic parameter mapping, bilinear interpolation can be used to project the boundary of the candidate touch area on the visible light side onto the infrared pixel grid, and a morphological closing operation is performed on the boundary to eliminate boundary holes caused by mapping, thereby making the avoidance mask coverage continuous.

[0040] First, the exposure time and analog gain of the infrared receiver are frozen and the timestamps are aligned. Then, candidate touch areas are obtained through the visible light side and mapped onto an avoidance mask. The avoidance mask removes the areas covered by hands and fingers from the environmental statistics. The main content of the environmental infrared interference fingerprint is the change of background rather than the touch action. The exposure time and analog gain are fixed within the sampling time window to make the infrared samples within the same time window comparable, which facilitates the subsequent judgment of background drift and periodic fluctuations.

[0041] The sampling time window is composed of several consecutive verification time windows. The boundaries of the sampling time window are only used for updating reliability records and loopback writing. The length of the verification time window... The boundary is only used for relevant demodulation and challenge-response verification, and the boundary between the two must not update the same parameter. The reliability record, encoding demodulation record, and touch verification record all contain a timestamp field and adopt the write-overwrite rule; when reading in step four, the timestamp must be completely consistent for it to be usable, otherwise the entry will be downgraded to a candidate to be kept and the next verification time window will be required to re-form the same source evidence.

[0042] To address common anomalies, strong edge reflections, and local bright spots in infrared received images, the samples are cleaned to stabilize the input for environmental feature calculation. The infrared samples are then statistically analyzed using a dual-channel approach: a full-screen background channel and a candidate touch area channel. This ensures that subsequent fingerprints can reflect the overall environmental intensity while preserving local interference near the candidate touch area.

[0043] In this process, after acquiring multiple frames of infrared received images within each sampling time window, the infrared receiver first performs pixel replacement processing on the infrared received images using a constant outlier table. The constant outlier table can be obtained by collecting data through light shielding when the equipment leaves the factory, or by covering the infrared receiver with a light shield during maintenance, and remains unchanged during equipment operation; the replacement method uses neighborhood median replacement, so that outliers do not form isolated bright spots or isolated dark spots.

[0044] An avoidance mask is applied to the infrared received image. Pixels covered by the avoidance mask are removed from the overall background channel statistics, leaving only pixels not covered by the avoidance mask to form the background statistical sequence. Simultaneously, a statistical sequence of candidate touch area channels is formed within the candidate touch area to characterize whether there is abnormal brightness, local saturation expansion, or local drift in a local area. For areas with strong edge reflections, an edge shielding band can be set with fixed geometric boundaries during device installation, and this edge shielding band is included in the removal set during the cleaning process to avoid persistent bias in the background statistics caused by reflections from the screen edge structure.

[0045] Specifically, based on a constant outlier list, abnormal pixels in the infrared received image undergo neighborhood median replacement and edge masking bands are added. An avoidance mask divides the overall image background channel into two forms: a full-screen background channel and a candidate touch area channel, resulting in a dual-channel statistical sequence. Abnormal pixel replacement and edge masking remove constant pseudo-features from the statistical sequence, allowing subsequent fingerprints to better reflect changes caused by ambient light and external infrared radiation, while preserving global and local background interference information. This provides a clear source explanation for subsequent reliability outputs.

[0046] Furthermore, using a dual-channel statistical sequence as input, fingerprint elements capable of describing the mechanism of environmental interference are constructed, enabling subsequent reliability outputs to have an interpretable causal source. Subsequently, phase-locked energy estimation is performed on background changes containing periodic fluctuations to obtain flicker energy used to characterize indoor lighting flicker or external modulated supplementary light, thereby distinguishing background fluctuations from random drift.

[0047] Specifically, for the statistical sequence of the background channels of the entire image, a background brightness index is constructed. This represents the overall background intensity value within the sampling time window; the proportion of construction saturation. This indicates the number of pixels in the infrared received image that have reached the saturation threshold; a drift index is constructed. This indicates the relative change in background brightness between adjacent sampling time windows.

[0048] For components with periodic fluctuations in the full-screen background channel statistical sequence, phase-locked projection is used to estimate the flicker energy. The statistical sequence of the background channel of the entire image is merged into a one-dimensional sequence within the sampling time window. and will Projecting these onto the sine and cosine basis functions respectively, we obtain the energy, in the following form:

[0049]

[0050] Where: scintillation energy Used to characterize the normalized frequency within the sampling time window. The corresponding periodic fluctuation intensity, with values... A larger value indicates a stronger frequency component; sampling sequence : A one-dimensional sequence obtained by aggregating the statistical sequences of the background channels of the whole screen. Each item corresponds to the background statistics of a frame or a segment within the sampling time window, and the value range is consistent with the quantization range of the infrared receiver.

[0051] Number of sampling points Within the sampling time window The number of terms, a positive integer, is constrained by both the frame rate of the infrared receiver and the length of the sampling time window; normalized frequency. : Used to specify the location of the periodic component to be estimated, with a value of This corresponds to the frequency position within the Nyquist range of the sampled sequence; in the exponent This indicates the amount of phase advance.

[0052] Furthermore, when the infrared receiver has sufficient computing power, discrete Fourier transform can be used to estimate multiple [samples] simultaneously within the same sampling time window. Candidates are selected from the frequency positions with the highest energy as the main flashing frequency; when computing power is limited, the above phase-locked projection can be performed only on a few frequency positions determined by the installation scenario.

[0053] for , , The saturation percentage can be calculated by segmenting and counting pixels using a pixel histogram, or by directly counting the saturation flags output by the sensor; the drift index can be calculated using adjacent sampling time windows. Normalized difference is performed to reduce the dimensional effects under different exposure durations.

[0054] First, a background brightness index is constructed based on a dual-channel statistical sequence. saturation ratio With drift index The scintillation energy was then calculated using phase-locked projection. This process generates an environmental infrared interference fingerprint. The fingerprint elements separately express background intensity, saturation state, time drift, and periodic fluctuations, enabling subsequent reliability outputs to distinguish between different interference mechanisms. Phase-locked estimation emphasizes the energy of specific periodic components, allowing scintillation-type interference to enter the fingerprint record in a stable manner, avoiding confusion with drift terms.

[0055] Furthermore, the elements in the environmental infrared interference fingerprint are first mapped to infrared reliability, so that the availability of the infrared outer side in the current sampling time window is output as a single marker.

[0056] The confidence level of key points, exposure state, and motion blur state on the visible light side are then mapped to visible light reliability. Both types of reliability, along with the environmental infrared interference fingerprint, are written into the reliability record for use as the sole input in subsequent steps. Infrared reliability... The saturation ratio decreases in a monotonically decreasing manner. Drift index Flashing energy Background brightness index The common constraints ensure that increased saturation, increased drift, enhanced flicker, and background rise can be directly reflected as a decrease in reliability. Specifically, this can be achieved through a combination of exponential decay and fractional suppression mapping.

[0057]

[0058] Where: Infrared reliability : Used to characterize the availability of infrared observations within the sampling time window, with values ​​ranging from 0 to 1. ; saturation ratio : Used to characterize the proportion of saturated pixels, with values ​​ranging from 0 to 10. ; Count the ratio of the number of pixels that reach the saturation threshold to the total number of pixels in the same pixel set; The saturation threshold is the maximum quantization value of the sensor or a fixed percentage threshold thereof;

[0059] Drift index : Used to characterize the relative magnitude of background brightness change between adjacent sampling time windows, with values ​​ranging from 0 to 1. Drift index Background brightness index of two adjacent sampling time windows Normalized differences are performed, and a numerical stability term is added to avoid the denominator being zero, as shown below:

[0060]

[0061] Historical background brightness index Background brightness statistics for the previous sampling time window, with a range of values ​​between Consistency; Drift Indicators : The normalized amplitude of background brightness variation with the sampling time window, with a value range of . Numerical stability term : Used to avoid positive numbers with a denominator of zero, the range of values ​​is It shares the same sign and meaning (numerical stability term) as the numerical stability term in peak-side ratio calculations. Scintillation energy : Energy used to characterize a specific periodic component, with values ​​ranging from 1 to 2. Background brightness index Used to characterize the overall background intensity level, its value range is consistent with the quantization range of the infrared receiver; the larger the value, the more obvious the background enhancement; it is taken as the pixel gray level. quantile value and the first The mean of the quantiles, where the quantiles are obtained by cumulative counting from a histogram;

[0062] Saturation weight Used for adjustment right The suppression strength, taking values The drift weight can be set during the installation and commissioning phase based on the sensor's dynamic range. Used for adjustment right The suppression strength, taking values The sampling time window length and background update strategy can be jointly set; flicker weight Used for adjustment right The suppression strength, taking values Brightness weighting Used for adjustment right The fractional suppression strength, taking values Used to reduce reliability when background is raised; fractional terms Used to introduce saturation suppression when the background is raised, avoiding insufficient sensitivity of a single exponential term to large background rises.

[0063] On the visible light side, visible light reliability The decision is based on key point confidence level, overexposure / underexposure status, and motion blur status: When key point confidence level decreases, resulting in overall overexposure / underexposure and noticeable motion blur at the fingertips, the visible light reliability is lowered according to the set rules. When the confidence level of the key points remains normal and the exposure is consistent, the visible light reliability is... It can reach a high level.

[0064] To facilitate direct determination of strategy branches in subsequent steps, infrared reliability... Reliability with visible light It can also be discretized into three levels of tags: high, medium, and low. The discretization threshold is set and fixed during the installation and debugging phase to avoid threshold drift during operation. Finally, the environmental infrared interference fingerprint, infrared reliability, and visible light reliability are written to the reliability record with the same timestamp, and an overwrite rule is set for the reliability record so that subsequent steps always read the three types of output corresponding to the latest sampling time window.

[0065] Key point confidence index : The minimum or average value of the confidence value of the fingertip keypoint within the time window (directly output by the model); Overexposure / Underexposure ratio index : The sum of the percentages of pixels in a visible light image that are above a bright threshold or below a dark threshold (the threshold is a fixed quantization threshold); Blur index : Calculate the reciprocal form of the sum of the absolute values ​​of the Laplacian operator in the fingertip neighborhood to reflect high-frequency attenuation; based on this, a mapping is given:

[0066]

[0067] Visible light reliability The availability of visible light observations, with a range of values. Key point confidence level index : Confidence statistic of key points on the fingertip, with a range of values ​​of [missing information]. Overexposure / underexposure ratio index The sum of the percentage of overexposed pixels and the percentage of underexposed pixels, with a value range of [value range missing]. Fuzzy Indicators : High-frequency attenuation metric in the fingertip neighborhood, with a value range of Overexposure and underexposure weighting Suppression coefficient, with a value range of [value range missing]. Fuzzy weights Suppression coefficient, with a value range of [value range missing]. .

[0068] First, infrared reliability is generated according to monotonic mapping. They were assigned high, medium, and low labels respectively; finally, a visible light reliability score was generated. It is also included in the reliability record along with the environmental infrared interference fingerprint, infrared reliability. The effects of saturation, drift, flicker, and background rise on infrared radiation are flagged, and environmental thresholds are applied during invocation; visible light reliability. The degree of confidence in the important points and the imaging quality are added as a marker, and the causes of infrared instability and visible light instability are distinguished when calling.

[0069] Step 2: Based on environmental infrared interference fingerprints and infrared reliability The transmission code sequence and transmission parameters are determined, and the detectability index of the actively encoded infrared component is obtained by matching correlation demodulation within the verification time window, thereby providing a unified source of evidence for the one-time verification code verification in step three and the gating fusion in step four.

[0070] Step 1 compresses the environmental state into an environmental infrared interference fingerprint, and then verifies the infrared reliability. It can be seen that infrared observation is available within the sampling time window; if step two still uses a brightness threshold or frame difference to eliminate background, then the background brightness... Improvement, saturation ratio Boost, Drift Boost Flashing energy Under enhanced operating conditions, touch evidence and background noise still overlap, and the threshold may drift over time.

[0071] Unlike direct thresholding of images, coded emission and matched correlation demodulation can separate the structural components of active emission from the uncoded background components, all within the same verification time window, avoiding the mechanism defect of residual differences between bright and dark frames during motion. Furthermore, ambient infrared interference is not a single intensity issue; the flicker energy in step one... and normalized frequency This indicates the presence of a periodic fluctuation in the background. If the transmitted code sequence in step two is superimposed on this periodic fluctuation in the spectrum, the related demodulation sidelobes may be pushed up by the periodic component, thereby reducing the peak-to-sidelobe ratio and causing spurious peaks.

[0072] Therefore, step two requires selecting the transmission code sequence based on the environmental infrared interference fingerprint, and considering infrared reliability. Bounding the code length, chip rate, duty cycle, and verification time window length ensures that the evidence from subsequent demodulation has a stable peak shape and an interpretable source.

[0073] Since step two will subsequently serve the one-time verification code verification within the candidate touch area in step three, the demodulation in step two must support sampling within the candidate touch area and output quality metrics that can be referenced by the gating decision; therefore, in generating the transmission code sequence At the same time, the reliability of visible light also needs to be considered. As a reference input for the sampling boundary of the candidate touch area, when overexposure, underexposure, or motion blur occurs on the visible light side, step two can tighten the sampling area boundary to a region closer to the fingertip, avoiding the introduction of irrelevant bright reflection areas into the relevant demodulation.

[0074] The environmental infrared interference fingerprint and infrared reliability output in step one A candidate codebook set is generated under constraints, and the code sequences in the candidate set are mapped to a feasible set of transmission parameters, ensuring that transmission and demodulation within the subsequent verification time window have clear computational boundaries. First, a computable relationship is established between the location of environmental scintillation components and the spectral sensitivity of the code sequence to avoid the sidelobe structure of subsequent related demodulation being raised by environmental fluctuations. Then, the code spectral collision amount is calculated for each candidate code sequence within the preset codebook, and a candidate code sequence set is formed accordingly. This eliminates the need for subsequent cost evaluation to be performed across the entire codebook, thus constraining the computational boundary to a reproducible candidate set.

[0075] In step two, the normalized frequency is first read from the reliability record. With flashing energy And determine each candidate transmission code sequence in the preset codebook. The rules for determining the value of the transmission code sequence. A bipolar sequence (with values ​​of...) can be used. and To enable demodulation using multiplication and accumulation, a unipolar sequence (with values...) can also be used. and At the receiving end, it is equivalently converted into a bipolar sequence through bias cancellation.

[0076] For each candidate transmit code sequence In code length Calculate its normalized frequency within the discrete index range. Code spectral collisions at the location This is used to characterize the projectibility of the code sequence with respect to environmental flicker components:

[0077]

[0078] Where: Code spectrum collision rate Used to characterize the transmit code sequence normalized frequency The projected energy at that location, taking values A larger value indicates that the code sequence is more likely to be projected and superimposed on the periodic fluctuations at that frequency; normalized frequency The periodic position parameter output from step one and written into the reliability record has the following value: , which corresponds to the frequency position within the Nyquist range of the sampled sequence;

[0079] Code length Transmit code sequence Discrete length, positive integer, length of the verified time window With chip rate Constraints; Transmit code sequence The first candidate sequence in the preset codebook Each code element takes a value, and the value is taken from... or Used to drive the transmit switch or phase flip of the infrared transmitter;

[0080] Furthermore, code spectrum collision quantity The calculation can be done by direct accumulation, or it can be performed on a fixed value during the offline codebook construction phase. The value grid is pre-calculated and stored as a code spectrum table; during device operation, only table lookups and interpolation are performed. Interpolation methods can be linear or cubic interpolation. Linear interpolation is easier to implement on low-computing-power controllers, while cubic interpolation is advantageous for applications requiring higher computing power. Maintain continuous changes in code spectrum collision amount during slight drift. Complete code spectrum collision amount. After the calculation, step two removes code sequences with a code spectrum collision rate higher than the preset collision threshold from the candidate set and forms a candidate code sequence set with the remaining code sequences.

[0081] Furthermore, first read the normalized frequency. And the transmission code sequence in the preset codebook Calculate code spectrum collisions Subsequently, candidate code sequence sets are selected based on the code spectrum conflict threshold and their indices are fixed.

[0082] Code Spectrum Collision Quantity By establishing a computable correspondence between environmental scintillation components and the spectral sensitivity of code sequences, the candidate code sequence set avoids major periodic fluctuations before entering subsequent cost evaluation. Pre-screening of the candidate code sequence set narrows the computational boundary to a reproducible set, ensuring that subsequent parameter delimitation does not depend on an uncertain global search process.

[0083] Let the code spectrum collision threshold be... (Fixed during installation and debugging phase), the candidate code sequence set is as follows:

[0084]

[0085] Candidate set The set of code sequences after code spectrum collision filtering, a finite set; threshold : Positive number, with a range of values ​​of It is used to control the flicker immunity strength of the reserved code sequence.

[0086] As a further step, the background brightness index in the ambient infrared interference fingerprint was first... saturation ratio Drift index Flashing energy Collision quantity with output code spectrum The overall cost is incorporated to determine the transmission code sequence that best matches the current environmental state among the candidate code sequences. Then, based on infrared reliability... For code length Chip rate Duty cycle Verification time window length With upper limit of transmission power Boundaries are defined to satisfy the sensor's dynamic range, temperature rise constraints, and eye safety constraints, and to provide boundary conditions for synchronous transmission sampling.

[0087] Step two involves pre-configuring sidelobe costs for each candidate code sequence. Side lobe cost The sidelobe cost is determined by the concentration of the autocorrelation sidelobes of the code sequence. The smaller the sidelobe cost, the easier it is for the code sequence to form a single-peaked correlation response.

[0088] Side lobe cost The codebook can be generated by enumerating shift-related responses and accumulating the absolute values ​​of non-zero shifts during the codebook generation stage, or it can be directly assigned using the theoretical sidelobe limits of the code family; sidelobe cost The specific calculation method can be achieved offline by using the sum of absolute values ​​of non-zero shift correlations, and the calculated result is... The code sequence index is stored together. Then, during device runtime, it will be... The combined cost of logarithmic compression of environmental infrared interference fingerprints Combined, this forms a single criterion for selecting the transmit code sequence:

[0089]

[0090] Where: Comprehensive cost : Used to evaluate the suitability of candidate code sequences under infrared interference fingerprints in the current environment, with values ​​ranging from 0 to 1. The smaller the value, the better the fit; numerical stability term : Used to avoid positive numbers with a denominator of zero, the range of values ​​is Sidelobe cost :

[0091]

[0092] The inherent sidelobe penalty of the candidate code sequence, with values ​​ranging from... Used to represent the autocorrelation sidelobe structure of the code sequence; code spectrum collision quantity : Value Used to represent the code sequence and normalized frequency Projected energy of periodic components; background brightness index The output from step one and written to the reliability record are used to characterize the degree of infrared background rise, and the value is consistent with the quantization range of the infrared receiver; saturation ratio. The output from step one is written to the reliability record, and the value is taken as follows: Used to measure the percentage of saturated pixels; drift metric The output from step one is written to the reliability record, and the value is taken as follows: , used to characterize the relative magnitude of background changes between adjacent sampling time windows;

[0093] Flashing Energy The output from step one is written to the reliability record, and the value is taken as follows: Used to characterize the intensity of periodic fluctuations; code spectrum weights Used for adjustment In terms of overall cost The intensity of the influence, taking values During the installation and commissioning phase, the brightness weight is set based on the sensitivity to the main flicker interference. Used for adjustment In terms of overall cost The intensity of the influence, taking values ; Drift weight Used for adjustment In terms of overall cost The intensity of the influence, taking values This is used to increase the preference for stable code sequences when the background changes rapidly; flicker weights Used for adjustment In terms of overall cost The intensity of the influence, taking values fractional terms Used for coupling background brightness index With saturation ratio ,when Increasing the denominator decreases the overall cost, thus amplifying the impact of background elevation on the overall cost. (Value...) ;

[0094] Furthermore, the overall cost Minimizing the value can be achieved by iterating through the candidate code sequence set, since the candidate set has already been determined after pre-screening; when the candidate set is still large, it can be minimized by considering the sidelobe cost. Sort the first few barcode sequences and then calculate the overall cost. This further limits the traversal scope to a manageable scale for the project.

[0095] Determine the overall cost After obtaining the minimum transmission code sequence, step two is based on infrared reliability. With saturation ratio Bounding the transmission parameter set: when infrared reliability When in a low setting, the code length will be... Set to the preset larger setting and set the verification time window length. With code length Alignment increases the processing gain of the relevant accumulation; when the saturation ratio is reached... When in a high position, the duty cycle will be... Lower and issue a lower transmit power limit This prevents the infrared receiver from further expanding its saturation region due to excessive emission under a fixed exposure time; when the scintillation energy When significant, the chip rate The setting is placed far from the main flicker period, ensuring that the main energy of the code sequence does not fall within the flicker frequency band. These settings can be predefined as a parameter set during the installation and commissioning phase using discrete enumeration. During device operation, the selection is limited to within this set, avoiding the introduction of an unreproducible continuous parameter tuning process.

[0096] During processing, the first step is based on the sidelobe cost. and comprehensive cost Select the transmission code sequence and fix its index; then, based on infrared reliability... saturation ratio With flashing energy Determine the code length within the preset gear set. Chip rate Duty cycle Verification time window length With upper limit of transmission power .

[0097] Overall Cost The inherent sidelobe structure of the code sequence is coupled with the environmental infrared interference fingerprint as a single selection criterion, giving the determination of the transmitted code sequence a clear input source and a traceable derivation path; the transmission parameter set is delimited in a discrete range and linked to infrared reliability. saturation ratio Flashing energy Establish a correspondence so that subsequent related demodulation still has a stable accumulation boundary in saturation and flicker scenarios.

[0098] First, the output transmission code sequence and transmission parameter set are converted into the switching timing of the infrared transmitter, ensuring that the transmission behavior strictly follows the code sequence beat within the verification time window. Then, the sampling beat of the infrared receiver is aligned with the transmission beat, and the received sampling sequence is extracted from the candidate touch area to generate a correlation response sequence, thereby expressing the active coded components in the form of correlation peaks.

[0099] Among them, the infrared transmitter sends the code length to the controller. Chip rate Duty cycle With the length of the verification time window Then, construct at chip rate The chip clock is a tick clock, and within each chip clock's effective interval, the duty cycle is... Control the infrared emission carrier to transmit the code sequence The transmission sequence is mapped to on-off or phase-flipped patterns. The infrared receiver operates within the same verification time window. The exposure time remains constant, and the received sampling sequence is extracted from the candidate touch area according to the sampling phase consistent with the chip clock. The boundary of the candidate touch area is determined by the avoidance mask from step one and the visible light reliability. Constraints: When visible light reliability At lower settings, the boundaries of candidate touch areas are narrowed to a smaller area within the fingertip projection neighborhood to reduce the probability of including highly reflective areas in the sample. Matched correlation demodulation generates a correlation response sequence in the form of discrete shift correlation. , where shift variables This indicates the alignment offset between the candidate touch area sampling sequence and the emission code sequence.

[0100] Shift operations use zero-padding shifts: when the index exceeds the code length The corresponding symbol is treated as zero to avoid boundary spurious peaks introduced by cyclic shifting. The relevant response sequence is calculated as follows:

[0101]

[0102] In the formula, the relevant response sequence Used to characterize the received sample sequence With the transmitted code sequence In shift The degree of matching, with values... The larger the absolute value, the higher the degree of matching; shift variable : Represents a discrete index for the alignment offset of the code sequence, taking the set of integer values, which can be limited to in engineering practice. A finite window is used to control the computational load, where It is a positive integer and is determined by the upper bound of the timing alignment error; receive the sampled sequence. : A sequence of discrete sampled values ​​extracted from the candidate touch area, with an index range of to The value range is consistent with the quantization range of the infrared receiver; the infrared camera uses a rolling shutter, and the controller controls the transmission switch in a line-synchronized manner within each chip cycle, so that the exposure of the same line covers the fixed chip phase, thereby forming equivalent symbol sampling in the line domain and converging them into a single signal. .

[0103] Transmit code sequence Transmit code sequence The index of the symbol value after zero-filling shift exceeds the limit. The time value is Code length : The determined length of the transmitted code sequence, a positive integer, and the length of the received sample sequence. The lengths are consistent;

[0104] Furthermore, the relevant response sequence The calculation can be implemented using direct accumulation; when the codebook uses the Walsh code family, the Fast Walsh-Hadamard Transform can be used to reduce the multiplication-addition complexity to logarithmic level, and within the candidate touch area, a fixed length can be used. The execution is performed in vector form; when the codebook uses a pseudo-random code family, the shift correlation can be implemented as discrete convolution and accelerated using Fast Fourier Transform, but to ensure the correctness of boundary zero-padding, the sequence length needs to be zero-padding to be no less than The length is calculated and the time domain is retrieved after multiplying in the frequency domain.

[0105] First, the transmit code sequence The transmission parameter set is mapped to the chip switching timing of the infrared transmitter and the sampling phase is locked; then the received sampling sequence is extracted from the candidate touch area. And calculate the relevant response sequence ; Related response sequence The actively encoded components are expressed as shift-correlation peaks, which suppress the uncoded background components during accumulation and form an interpretable form of evidence. Zero-filling shifts constrain boundary behavior, ensuring that the formation of correlation peaks is mainly determined by the matching of the true code sequence, thus reducing the risk of spurious peaks caused by cyclic boundaries.

[0106] Furthermore, from the relevant response sequence The quality indicators directly related to touch confirmation are extracted to enable subsequent steps to make gating decisions on the reliability of relevant peaks. These quality indicators, along with the determined transmission parameter set, are then written into the encoding and demodulation record. The record is kept aligned with the reliability record from step one via timestamps to ensure that steps three and four do not involve cross-time window misuse. Step two first determines the relevant peak candidate window and the sidelobe window: the relevant peak candidate window covers the allowable range of alignment offset, and the sidelobe window covers other shift ranges besides the peak candidate window.

[0107] The candidate window for the relevant peak can be constructed around the theoretically aligned offset center, which is determined by the trigger delay and exposure phase of the infrared transmitter and receiver; the sidelobe window is then taken as its complement. The peak-sidelobe ratio is then constructed. As a single quality indicator, it is used to characterize the separation between the relevant peak and the sidelobes, and is expressed by a small positive number. Numerical anomalies caused by limiting the denominator to an extremely small value. Peak-side ratio. The specific form is as follows:

[0108]

[0109] In the formula: peak-side ratio : Used to characterize the degree of separation between the relevant peak and the sidelobes, with values ​​ranging from 0 to 1. ; Related response sequence : The generated shift-related response, with values... Peak candidate window Used to limit the search range of relevant peaks, its value is a shift variable. A subset of the integer set, the window width is determined by the upper bound of timing alignment error and the upper bound of trigger jitter; sidelobe window Used to limit the search range of sidelobes, its value is a shift variable. A subset of the set of integers that satisfies and Non-overlapping and covering the computable shift range except for the peak candidate window; small positive number Used to avoid numerical instability caused by a denominator of zero or extremely small values; [Value not specified] It is fixed during the installation and commissioning phase.

[0110] Beside the peak In addition, step two simultaneously generates a peak consistency index: first in the peak candidate window The shift with the largest absolute value is taken as the peak position, and this peak position is compared with the theoretical alignment offset center. If the deviation exceeds the allowable offset, it is marked as peak position inconsistency. Peak shape stability can be verified by the length of two adjacent verification time windows. Internal repeat calculation of peak-side ratio It also requires that the peak-to-side ratio be kept at the same level.

[0111] When the aforementioned peak position consistency and peak shape stability do not introduce new mathematical symbols, they can still be written as discrete markers in the coded demodulation record. Finally, the coded demodulation record is simultaneously written into the transmit code sequence. Index, code length Chip rate Duty cycle Verification time window length Maximum transmit power Peak-side ratio The peak position consistency marker and peak shape stability marker are linked to the reliability record in step one through the same timestamp to ensure that subsequent steps read consistent input according to the time window.

[0112] Specifically, firstly, based on the relevant response sequence Constructing peak candidate windows With sidelobe window And calculate the peak-side ratio. Subsequently, peak position consistency markers and peak shape stability markers are generated and written into the encoding and demodulation record along with the transmission parameter set.

[0113] Peak side comparison The relevant peak prominence is compressed into a single quality metric, enabling subsequent steps to reference relevant demodulation evidence with explicit gating conditions. The encoded demodulation record timestamps the quality metric and transmission parameter set, ensuring that steps three and four have the same source input when invoked and avoiding ambiguity in decisions caused by mixing across time windows.

[0114] Peak position selection Maximum shift:

[0115]

[0116] when When peak position consistency is consistent, it is marked as consistent; otherwise, it is inconsistent. Peak shape stability marking follows the cross-window consistency rule: continuous... Within a verification time window ( (For the installation and commissioning phase, solidify the positive integer peak-side ratio) If the threshold is met and the peak position consistency is marked as consistent, then the peak shape stability is marked as stable; otherwise, it is unstable.

[0117] Step 3: Within the candidate touch area triggered by visible light, a challenge-response process involving a one-time verification code transmission and related demodulation is used to confirm valid touch, and a touch verification record that can be referenced by subsequent gating fusion is output.

[0118] Step 1: Using ambient infrared interference to characterize the fingerprint background brightness index saturation ratio Drift index Flashing energy with normalized frequency And use infrared reliability Reliability with visible light The availability of two types of observations is given; step two determines the launch parameter set based on this and the verification time window length. Internal generation of relevant response sequences Compared with the peak .

[0119] However, under conditions of direct sunlight, top supplemental lighting, and infrared emission from adjacent terminals, the infrared receiver may experience phenomena such as movement of localized high-brightness reflection areas, superposition of periodic changes, and instantaneous saturation expansion, resulting in certain time-related response sequences. In the peak candidate window There are occasional peaks in the vicinity. If only fixed criteria are used, it is difficult to distinguish between occasional peaks and the relevant peaks generated by actual touch. This will result in accidental touches and jitter on the business interface.

[0120] Therefore, step three needs to advance the touch confirmation from a one-time threshold judgment to a challenge-response verification. That is, when the visible light side obtains a candidate touch area and meets the triggering conditions, the infrared transmitter emits the transmission code sequence corresponding to the one-time verification code within the same verification time window. The infrared receiver only matches relevant demodulation within the candidate touch area, and multiplexes the peak-side ratio. With peak candidate window The consistency constraint combines the geometric evidence triggered by visible light with the same code evidence on the infrared outer side into a single-link decision, and provides interpretable input for the risk-sensitive fusion in step four.

[0121] Read the reliability record and code demodulation record to lock the transmission parameter set (code length) used in this verification. Chip rate Duty cycle Verification time window length Maximum transmit power ), and cited infrared reliability Reliability with visible light Set the boundaries of the candidate touch area and the candidate touch trigger conditions.

[0122] Subsequently, the visible light side observations of the hand and fingertips are converted into candidate touch areas, and the candidate touch areas are mapped onto the infrared receiving image to determine the sampling boundary. Then, when the candidate touch triggering conditions are met, a one-time verification code verification task is assembled, so that the task has a clear start and end time window and a clear area boundary.

[0123] After the verification task is triggered, the transmission code sequence corresponding to this one-time verification code is selected from the candidate code sequence set pre-selected in step two. and in the verification time window length The infrared receiver completes the transmission and reception sampling within the candidate touch area; the infrared receiver calculates the relevant response sequence within the candidate touch area. And derived peaks beside Combined with infrared reliability With flashing energy Provide an interpretable verification threshold, and finally write the verification conclusion, evidence indicators and associated timestamps into the touch verification record and output them for use in step four.

[0124] Starting with observations of the hand and fingertips on the visible light side, the human input location is converted into an executable candidate touch area, which is then used as the unique spatial boundary for sampling by the infrared receiver. Once this spatial boundary is locked, subsequent demodulation will not indiscriminately accumulate across the entire image, thus avoiding the introduction of bright reflective areas far from the fingertips into the relevant response sequence. The formation process.

[0125] In the visible light side, the hand contour and fingertip position are generated in each sampling time window. The initial boundary of the candidate touch area is generated with the fingertip position as the center. The initial boundary is then determined based on the visible light reliability. Tighten or loosen; when visible light reliability When the reliability is zero, the candidate touch area only includes the area near the fingertip, and the boundary of the candidate touch area must remain continuous in a continuous visible light frame; when the visible light reliability is zero... The high-time candidate touch area includes the natural swing margin of the fingertip and the ends of other adjacent fingers to avoid sample loss.

[0126] When mapping the candidate touch area from visible light coordinates to the infrared receiving screen, the extrinsic parameter mapping relationship used in step one is reused, and bilinear interpolation is used to project the boundary onto the infrared pixel grid. If there are discrete breaks in the boundary after mapping, a morphological closing operation is performed on the boundary to fill the breaks, ensuring that the candidate touch area forms a continuous closed region on the infrared receiving screen. This closing operation uses fixed structural elements and is solidified during the installation and debugging phase to ensure that the result is reproducible. After mapping, the infrared receiver registers the candidate touch area as the sampling boundary, and only pixel values ​​within the sampling boundary are allowed to enter the receiving sampling sequence. This constrains the relevant response sequence. The source of.

[0127] Specifically, firstly, according to the reliability of visible light Candidate touch areas are generated, ensuring their continuity within the visible light frame. These candidate touch areas are then mapped onto the infrared receiving frame, and their sampling boundaries are registered. The candidate touch areas confine infrared sampling to the vicinity of the fingertip, thereby enabling the relevant response sequence... Form a set of pixels in the same region as the touch area. The boundary of the candidate touch area is reliable in visible light. To maintain continuity under constraints, so as to prevent boundary jumps from causing inconsistencies between the sampling boundary and adjacent time windows.

[0128] Candidate touch trigger conditions are then generated based on the candidate touch areas, and the verification time window length is then set. The start and end boundaries enable challenge-response verification to be initiated at the appropriate time and kept in sync with subsequent launch timing.

[0129] This timing has a significant impact on the validity of one-time verification codes: if verification is performed too early, the fingertip is not stable near the candidate touch area, resulting in a different response sequence. Peak position falls within peak candidate window Additionally; if verification is performed too late, and the user has already clicked to leave, then the verification time window length will be adjusted. The valid information segment is compressed.

[0130] The test utilizes the change in fingertip position within a continuous visible light frame: the candidate touch trigger condition is met when the fingertip reaches the candidate touch area and remains within that area for a preset number of consecutive frames, and there are no large overexposed or underexposed areas on the visible light side. The preset frame number is a positive integer after installation, debugging, and setup, and is based on the length of the verification time window. For the smallest unit (e.g., needing to be continuous) (A verification time window is required to trigger this). Meanwhile, it is recommended that candidate touch area identity markers be generated using a deterministic method: the nearest neighbor association of fingertip keypoints in adjacent visible light frames is used as the generation method. If multiple candidate fingertips exist in the same frame, they are matched using the minimum displacement and hand contour consistency constraint. If the match fails, a new identity marker is generated.

[0131] The trigger condition reads the length of the verification time window that has been fixed in the encoding and demodulation record at the moment of triggering. Code length Chip rate Duty cycle And set the start time of the verification time window to the next chip clock boundary after the trigger instant, so that the transmit code sequence The first symbol is aligned with the sampling phase of the infrared receiver. If the infrared reliability... If the setting is low, the parameter set already fixed in step two is not changed. Instead, additional repetitive trigger suppression is added to the timing assembly: the same candidate touch area is within one verification time window. The second verification will not be triggered repeatedly within the saturation ratio, thus avoiding the situation where the saturation ratio is reached. Frequent transmissions in high-temperature environments cause further saturation expansion at the infrared receiver.

[0132] Specifically, candidate touch trigger conditions are first generated based on the stability of the fingertip position within the candidate touch area; then the verification time window length is... The start and end boundaries are aligned to the chip clock boundaries and repeated triggering within the same window is suppressed. The triggering condition limits one-time CAPTCHA verification to the period when the fingertip enters and remains stable, making it easier for subsequent related peaks to fall into the peak candidate window. Within the allowed range. Verify that the time window is aligned with the chip clock to reduce the transmission code sequence. With the received sampling sequence Phase mispairing related response sequence The impact.

[0133] Without altering the boundaries of the emission parameter group already fixed in step two, a corresponding one-time verification code is generated for each candidate touch trigger, and this one-time verification code is then implemented as a transmission code sequence that can directly drive the infrared transmitter. This implementation process requires meeting two constraints: first, the transmission code sequence. The candidate code sequences must come from the set of candidate code sequences pre-screened in step two, ensuring that their code spectrum collision rate and sidelobe structure satisfy the constraints of the current environment's infrared interference fingerprint; secondly, the transmission code sequence The selection of the infrared source should vary with the verification task to make it difficult for the external infrared source to form a stable peak in the relevant domain by means of fixed modulation.

[0134] The process involves reading the index of the candidate code sequence set embedded in the encoding / demodulation record and using the timestamp of the verification time window as a one-time selection input. The controller reads the key from secure storage, splits the timestamp byte-by-byte, XORs it with the key byte-by-byte, and then performs a fixed number of rounds of cyclic shifting and byte permutation on the XOR result to obtain an integer sequence. The controller then uses this integer sequence as a modulo operation with the length of the candidate code sequence set to obtain the index position within the candidate code sequence set, and selects the transmission code sequence accordingly. The above combination of XOR-cyclic shift-byte permutation-modulo can be performed by a general-purpose microcontroller before the start of the verification time window in engineering practice, and each step is a deterministic operation, ensuring that the same key corresponds to a unique transmission code sequence index with the same timestamp.

[0135] If the device lacks secure storage, the key can be replaced by a fixed sequence written at the factory, and a counter can be used to replace the timestamp input. If the device needs to process multiple candidate touch areas simultaneously, a different candidate code sequence set index window can be assigned to each candidate touch area, and the verification time window length can be specified. The internal chip clock is interleaved for transmission, enabling different candidate touch areas to distinguish their respective transmission code sequences during demodulation. The chip rate determined in step two is strictly followed during the launch assembly phase. Duty cycle With upper limit of transmission power and the selected transmission code sequence This is mapped to the switching timing of the infrared transmitter, thereby ensuring that the challenge-response transmission and the demodulation parameters in step two have consistent boundaries.

[0136] Specifically, the transmission code sequence within the candidate code sequence set is first determined using the timestamp and key through XOR, circular shift, byte permutation, and modulo operations. Subsequently, the chip rate was continued. Duty cycle With upper limit of transmission power Will This is mapped to the infrared transmitter switching timing sequence. Transmission code sequence. The selection is constrained by the candidate code sequence set to ensure that its code spectrum characteristics are consistent with the environmental infrared interference fingerprint, and to avoid introducing code sequences that strongly project onto the scintillation component. Transmit code sequence As the verification task changes, it becomes difficult for external infrared sources to form stable identical code evidence in the relevant domain with fixed modulation.

[0137] Using relevant evidence within the candidate touch area as the core, the response to the one-time verification code is judged, and the evidence and conclusion relied upon in the judgment process are written into the touch verification record. This writing serves two purposes: one is to provide a direct input for step four to determine whether to confirm the touch, and the other is to provide a chain of evidence for subsequent review to explain why the touch was confirmed or rejected, enabling gating fusion to use source evidence for state transitions.

[0138] Among them, the infrared receiver has a verification time window length Only candidate touch areas are sampled to form a received sampling sequence. The relevant response sequence is obtained by using the relevant calculation method defined in step two. Then, according to the peak candidate window With sidelobe window Calculate peak-side ratio .

[0139] To couple the decision threshold with the environmental state, infrared reliability will be... With flashing energy A threshold expression for the peak-side ratio is jointly introduced to improve infrared reliability. Decreasing or flashing energy During enhancement, the decision threshold is automatically raised to reduce false positives caused by occasional peaks. This threshold is expressed as follows:

[0140]

[0141] In the formula: peak-side ratio : A quality index used to characterize the degree of separation between relevant peaks and sidelobes, with values ​​ranging from 0 to 10. Flashing energy Step 1: Output and write the periodic fluctuation energy into the reliability record, and set its value. Small positive numbers Step two has defined and written the numerical stability term into the encoding and demodulation record, and its value is... This is used to avoid numerical anomalies caused by an extremely small denominator; infrared reliability. Step 1 outputs and writes the infrared availability level into the reliability record, with the value set to... The smaller the value, the greater the impact of saturation, drift, and flicker on infrared observation;

[0142] If the above inequality is satisfied, further check the peak position consistency: in the peak candidate window Within the specified timeframe, the relevant peak positions are determined and required to fall within the allowed alignment offset range. If the peak position deviates from the peak candidate window... If the touch confirmation is rejected, the response will be rejected. If short-term jitter needs to be suppressed, the peak-side ratio of the previous verification time window will be read again. Mark and require a threshold to be met for a preset number of consecutive times to avoid triggering confirmation by a single occasional peak.

[0143] Finally, the transmission code sequence used in this verification will be... Index, Transmission Parameter Group (Code Length) Chip rate Duty cycle Verification time window length Maximum transmit power Peak-side ratio The peak consistency marker, verification conclusion, and timestamp aligned with the reliability record are written into the touch verification record, and the touch verification record is then read by step four.

[0144] Specifically, firstly, based on the inequality threshold, the peak-side ratio is... Make a judgment and combine it with the peak candidate window Check peak consistency; then transmit code sequence The emission parameter set and judgment evidence are written into the touch verification record and output; the judgment threshold will determine the infrared reliability. With flashing energy Directly mapped to peak-side ratio The allowable conditions make it difficult to confirm occasional correlation peaks in strong interference scenarios; the touch verification record saves both conclusions and evidence, ensuring that step four can perform gating fusion and hysteresis state machine transition with the same source evidence and has traceable basis.

[0145] Step 4: Merge the touch verification record and the reliability record into a gating decision input, output a stable touch event, and write back the transmission parameter group and reception parameter of the next verification time window at the boundary of the sampling time window.

[0146] Step 1 provides the environmental infrared interference fingerprint (background brightness index). saturation ratio Drift index Flashing energy Normalized frequency This describes the intensity, drift, and periodic fluctuations of the infrared background; infrared reliability. Reliability with visible light The availability of the two types of observations is further compressed into a single time window.

[0147] Step 2: Determine the transmission code sequence accordingly. With the launch parameter set, and through the relevant response sequence Compared with the peak Provide structured evidence. Step 3 involves completing a one-time CAPTCHA challenge-response verification within the candidate touch area and writing it into the touch verification record, thereby merging the same-code evidence with the geometric trigger.

[0148] In field deployments with both high illumination and external infrared supplementary lighting, two types of boundary situations may still arise: First, although the touch verification record indicates effective touch, the infrared reliability... A decline occurred, peak-to-peak ratio Boundary fluctuations can cause repeated entry and exit of touch events within adjacent verification time windows; secondly, visible light reliability During descent, the spatial boundary jitter of the candidate touch area will affect the relevant response sequence. The peak position in the peak candidate window The peak consistency flag fluctuates near the boundary, causing it to oscillate between adjacent time windows. Therefore, step four requires further incorporating the verification conclusions into the risk-sensitive gating, stabilizing the touch event output through a hysteresis state machine, and then writing back the impact of environmental infrared interference fingerprints on the transmission and reception parameter sets in a closed-loop manner to match the transmission and demodulation boundaries of the next verification time window with the current environment.

[0149] Furthermore, for the phenomenon of different evidence appearing in different time windows for the same candidate touch area, the timestamps of the reliability record, encoding / demodulation record, and touch verification record are first aligned. Then, the evidence corresponding to the same timestamp is aggregated into a gated input set, thereby avoiding peak-side comparisons between different time windows. Infrared reliability Visible light reliability The decision drift caused by mixing different input methods. Subsequently, the gating input set is split according to the candidate touch area dimension, so that each candidate touch area has an independent gating input set entry, which facilitates the maintenance of the state region by region in the hysteresis state machine.

[0150] Specifically, the controller first reads the infrared reliability from the reliability record at the end of the sampling time window. Visible light reliability saturation ratio Background brightness index Drift index Flashing energy with normalized frequency Then read the peak-side ratio corresponding to the same timestamp in the encoded demodulation record. Peak candidate window Sidelobe window small positive numbers Verification time window length and the transmission parameter set (code length) Chip rate Duty cycle Maximum transmit power ).

[0151] Subsequently, the controller reads the verification conclusion and peak consistency flag in the touch verification record, and checks whether the timestamp of the touch verification record is consistent with the reliability record and the encoding / demodulation record. If the timestamps of the three are inconsistent, the touch verification record within that time window is considered unusable for touch confirmation, and the gated input set entries are kept as candidates. The next time window will generate the same source evidence again. When splitting the candidate touch area dimension, the controller uses the candidate touch area identity tag registered in step three (generated by the association between the hand and fingertip given by the visible light side) to form a continuous association between the gated input set entries of the same candidate touch area within the continuous sampling time window. When the candidate touch area identity tag is replaced, the controller puts the original candidate touch area entry into the release pending confirmation branch to avoid directly outputting the release event when the identity jumps.

[0152] First, it uses timestamp consistency checks to align reliability records, encoding / demodulation records, and touch verification records, eliminating cross-window misuse; then, it forms gating input set entries based on candidate touch area identity tags while maintaining cross-window association. Timestamp alignment and cross-window misuse elimination are used to compare peaks and side peaks. Infrared reliability Visible light reliability Under the same environmental infrared interference fingerprint constraint, gating decisions are prevented from fluctuating due to inconsistent evidence. The dimensional segmentation of candidate touch areas allows gating decisions and subsequent hysteresis state machines to be executed separately for each area, avoiding crosstalk of touch events caused by the overwriting of evidence in different areas.

[0153] Based on the gated input set entries, the verification conclusions, peak consistency markers, and peak-side ratios of the touch verification records are combined. Unified inclusion of gating decisions and infrared reliability Visible light reliability With flashing energy Explicitly coupled to peak-side ratio The threshold provides a clear source for changes in the threshold under strong interference scenarios.

[0154] With saturation ratio Background brightness index As a conservative mode switching factor, it determines whether to enter the branch of prioritizing candidates and strictly controlling contact confirmation, in order to adapt to interactive interfaces such as transaction confirmation and gate release where the cost of accidental touch is higher.

[0155] The gating decision first obtains the verification conclusion of the touch verification record: if the verification conclusion is negative, then no peak-to-peak comparison is performed. If an inference is made and a contact rejection is output, and the verification result is a valid touch, then the peak position consistency flag is further checked. Only when the peak position consistency flag is consistent will the peak side comparison be performed. Threshold determination.

[0156] Threshold determination uses infrared reliability Visible light reliability With flashing energy Coupling criteria:

[0157]

[0158] In the formula: peak-side ratio Step 2: Define and write the relevant peak prominence index into the encoding and demodulation records, and set its value. Used to characterize peak candidate windows Inner peak value and sidelobe window The degree of separation of the largest inner sidelobe; scintillation energy Step 1: Extract and write the periodic fluctuation intensity into the reliability record, and set the value... Used to characterize the intensity of the periodic components of indoor lighting flicker or external infrared illumination; small positive numbers Step 2: Write the numerical stability term into the encoding and demodulation record, and set its value. Used to ensure the denominator has infrared reliability It retains a positive value even when taking a low value and limits the amplification of extreme values;

[0159] Infrared reliability Step 1 outputs and writes the infrared observation availability into the reliability record, with a value of [value missing]. The smaller the value, the stronger the impact of saturation, drift, or flicker on infrared observation; in this judgment formula, infrared reliability... Entering the denominator, thus improving infrared reliability. As the threshold rises during a descent, a higher peak lateral distance is required. Only then can contact be confirmed.

[0160] Visible light reliability Step 1 outputs and writes the usability of visible light observations into the reliability record, with a value of [value missing]. The smaller the value, the more severe the overexposure, underexposure, or motion blur; in this judgment formula, because exist The top is not correct, the top is not correct. Follow The threshold increases as the visible light threshold decreases and increases, causing the visible light threshold to rise when the visible light is unstable.

[0161] When the above judgment formula is satisfied, the contact confirmation is output; when the judgment formula is not satisfied but the verification conclusion of the touch verification record is valid touch and the peak position consistency is marked as consistent, the candidate is kept and the entry is handed over to the hysteresis state machine for continued observation to avoid contact jitter caused by a single boundary fluctuation.

[0162] Switching to conservative mode will saturate the proportion Background brightness index As input: saturation percentage Recorded as high or background brightness index If the position remains high, the output of contact confirmation will be limited to satisfying the above judgment formula within multiple verification time windows, and the maximum duration of the candidate will be limited to the number of time windows set. If the contact confirmation is not satisfied after a certain number of verification time windows, the contact will be rejected, thus avoiding long-term suspension in saturation expansion scenarios.

[0163] First, based on the verification conclusions of the touch verification records and the peak consistency markers, available items are selected and a threshold judgment formula is calculated. Then, based on the saturation ratio... Background brightness index Switch to conservative mode and output gating decisions for contact confirmation, candidate hold, and contact rejection. The threshold decision method assesses infrared reliability. Visible light reliability With flashing energy Compared with the peak Direct coupling ensures that the criteria for contact confirmation have a clear and traceable source as the environment changes. Switching to a conservative mode will reduce the saturation rate. Background brightness index By adding a gating strategy branch, gating decisions in saturated expansion scenarios tend to favor candidate retention, thus suppressing false confirmations.

[0164] A hysteresis state machine is maintained for each candidate touch area entry, transforming the gating decision result into state transitions of hover, candidate, contact, and release, and outputting touch events at the state transition boundaries. Subsequently, different continuous satisfaction requirements are set for entering and exiting contact, so that the same candidate touch area is compared with peak-side conditions. Boundary fluctuations, infrared reliability Oscillation, Visible Light Reliability It can still output a stable sequence of events when sliding down.

[0165] Among them, the hysteresis state machine is used to verify the length of the time window. As the smallest temporal granularity: when the state machine is in the hover state and the gating decision is candidate hold, the state machine transitions to the candidate state and records the candidate start time window; when the state machine is in the candidate state and the gating decisions continuously satisfy contact confirmation, the state machine transitions to the contact state and outputs a contact event; when the state machine is in the contact state and the gating decisions continuously satisfy contact rejection, the state machine transitions to the release state and outputs a release event; when the state machine is in the release state and the gating decisions are continuously contact rejection, the state machine returns to the hover state. To form hysteresis, the number of consecutive contact confirmations required to enter the contact state and the number of consecutive contact rejections required to exit the contact state are set to different values ​​and are fixed during the installation and commissioning phase.

[0166] When the conservative mode is triggered, the maximum duration of the candidate state is further limited to a fixed number of verification time windows. If this number is exceeded, the candidate state is forcibly returned to the hover state to prevent the downstream interface from remaining in a half-triggered feedback state due to prolonged candidate state inactivity. For multi-finger or multi-hand input, the state machine is maintained separately using the candidate touch area identity marker as an index. Furthermore, when the boundaries of two candidate touch areas overlap, the peak-to-side ratio is preferentially preserved. Entries with higher peak consistency and consistent flags enter the contact state, while the other entry remains a candidate, thus avoiding duplicate output of two contact events at the same location. Specifically, this is first determined by the length of the verification time window. Hysteresis state machine transitions are performed at the candidate touch area dimension with temporal granularity. Contact and release events are then output at the state transition boundaries, and candidate state timeout fallback and overlapping entry arbitration are handled. The hysteresis state machine absorbs short-term fluctuations in gating decisions within the state, ensuring that touch event outputs have clear boundaries for entering and exiting contact, preventing rapid and repeated triggering on the interface. Candidate state timeout fallback and overlapping entry arbitration maintain an interpretable event sequence even in multi-user, rapid operation scenarios, preventing repeated output of contact events at the same location.

[0167] Based on the environmental infrared interference fingerprint and the gating decision result, a closed-loop adjustment command is generated to adjust the transmission parameter group (code length). Chip rate Duty cycle Verification time window length Maximum transmit power The next time window is selected based on the exposure time of the infrared receiver, the analog gain, and the boundary of the candidate touch area. The selection result is written back to the encoding and demodulation record so that the updated parameter boundary can be read at the beginning of the next verification time window in step two.

[0168] Subsequently, the effective boundaries of the parameters are limited to the sampling time windows to avoid the same verification time window length. Internal parameter jump failure related response sequence Comparability.

[0169] Among them, the closed-loop adjustment first works in the saturation suppression branch: when the saturation ratio is... High brightness or background brightness index When the level remains consistently high and gated decisions repeatedly result in candidate hold-up, the duty cycle... Update and adjust the duty cycle. The write-back is the duty cycle setting closest to the updated value. The update relationship uses the saturation percentage. Background brightness index Multiplicative relations with monotonic inhibition:

[0170]

[0171] Where: duty cycle Step 2: Determine and write the transmit duty cycle into the encoding / demodulation record. The value is a member of the duty cycle set defined during the installation and debugging phase. The duty cycle set includes the code length. Chip rate Verification time window length Duty cycle Exposure time and analog gain are given as discrete sets (the set members are a finite number of fixed values). When writing back in the closed loop, the rule of rounding down to the nearest integer and clipping to the boundary of the set is adopted.

[0172] Assignment symbol : Parameter update symbol, indicating that the calculation result on the right is written to the duty cycle. The update buffer is applied and the replacement takes effect at the start of the next sampling time window; saturation weights The saturation weights defined in Step 1 have the following values: This setting, implemented during the installation and commissioning phase, is used to adjust the saturation ratio. duty cycle The impact of the update magnitude; saturation percentage Step 1 outputs and writes the saturated pixel percentage to the reliability record, with the value set to... Brightness weighting The brightness weights defined in Step 1 are set to specific values. This setting, implemented during the installation and debugging phase, is used to adjust the background brightness index. duty cycle The impact strength of the update magnitude;

[0173] Background brightness index Step 1 outputs the background intensity index written into the reliability record, with the same value as the infrared receiver; duty cycle. The updated infrared receiver parameters are as follows: exposure time and analog gain are equal to the preset high light level during installation and debugging, and saturation ratio. Consistent levels; drift index When the size is too large, the boundary of the candidate touch area is tightened to the vicinity of the fingertip projection, increasing the number of consecutive confirmations to enter the touch state and avoiding drift that causes the peak position to be at the peak of the candidate window. It bounces at the boundary. Number of consecutive confirmations. Define the number of times to enter contact and the number of times to exit contact. For installation, debugging, and fixing, a positive integer is specified. This does not constitute hysteresis.

[0174] In the evidence enhancement branch, when infrared reliability... Medium setting and flashing energy It is at a low level, but the gating decision remains in the candidate hold state for a long time and no longer reduces the duty cycle. Instead, the code length With the length of the verification time window Switch to a longer range to allow the related accumulation to cover more symbols; simultaneously, increase the chip rate. Limited to the set of gears with low code spectral collisions in the candidate set of step two, so that the relevant response sequences Sidelobe window Not easily pushed up by periodic components. Code length Verification time window length Chip rate The gear selection is performed using discrete enumeration, which is conducted within a finite set of gears. In engineering practice, binary search can be used to locate the shortest verification time window length that satisfies the gating decision within the sorted set of gears. This allows for control of interaction latency.

[0175] All closed-loop adjustment results are written back to the update buffer of the encoder-demodulator record, with the start of the next verification time window as the effective boundary: the update buffer is only overwritten into the effective area when the controller detects that the current verification time window has ended, thus avoiding the repetition of the same related response sequence. The parameters change during the generation process.

[0176] Specifically, firstly, based on the saturation ratio Background brightness index duty cycle Perform multiplicative suppression update and write back to the duty cycle level set; then update the exposure duration, analog gain, and code length in conjunction at the sampling time window boundaries. Verification time window length Chip rate The boundary of the candidate touch area is defined, and the encoded demodulation record is written back in the form of an updated buffer.

[0177] Duty cycle The multiplicative suppression update will saturate the proportion Background brightness index The impact is explicitly mapped to the emission activation ratio, thus constraining the dynamic range occupancy caused by emission superposition in strong light saturation scenarios. The write-back method, using the sampling time window as the effective boundary, maintains the same verification time window length. With constant intrinsic parameters, the relevant response sequence The peak is more It is comparable among peers and facilitates gating decisions and state machine references.

[0178] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0179] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0180] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0181] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0182] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A robust multi-source fusion recognition method for infrared and visible light touch control, characterized in that: include, During device operation, infrared reception data is collected in short time windows to calculate the environmental infrared interference fingerprint and extract background brightness, saturation ratio, short-term drift and flicker components; at the same time, visible light hand and fingertip information is collected, and infrared reliability and visible light reliability are summarized and recorded. Based on the environmental infrared interference fingerprint and infrared reliability, the transmission code sequence and transmission parameters are selected from the preset codebook, and the transmission code sequence is transmitted within the verification time window; the receiving end samples in the interaction area and performs matching correlation demodulation, and outputs the correlation demodulation quality index; The visible light side generates candidate touch areas and provides candidate touch confidence levels based on visible light reliability triggering, and transmits a one-time verification code sequence; the receiving end only performs matching-related demodulation within the candidate touch areas and matches the one-time verification code sequence to form a touch verification record; Based on the gating fusion rules, the relevant demodulation quality indicators, candidate touch confidence, infrared reliability and touch verification records are judged and the touch event is output through the hysteresis state machine. The transmission parameters and the receiving exposure and gain are updated according to the environmental infrared interference fingerprint.

2. The robust multi-source fusion recognition method for infrared and visible light touch as described in claim 1, characterized in that: Within each short time window, statistics are collected on the full screen of the interactive area and the suspected touch neighborhood of the visible light prompts to generate an environmental infrared interference fingerprint. The infrared reliability and visible light reliability are represented by continuous values ​​with level labels, and written into the reliability record along with the timestamp as the input basis for steps two and four.

3. The robust multi-source fusion recognition method for infrared and visible light touch control as described in claim 2, characterized in that: The environmental infrared interference fingerprint includes background brightness, saturation ratio, short-time drift and flicker components. The flicker component is synthesized by projecting the statistical sequence of background brightness within a short time window onto the sine basis function and cosine basis function respectively, and extracting the dominant frequency and harmonic energy, and writing the dominant frequency position into the reliability record.

4. The robust multi-source fusion recognition method for infrared and visible light touch control as described in claim 3, characterized in that: In the preset codebook, the code spectrum collision amount is calculated for each candidate transmission code sequence according to the main frequency of the scintillation component, and the candidate code sequence set is formed by combining the sidelobe cost. Then, the transmission code sequence and transmission parameters are determined from the candidate code sequence set based on the infrared reliability and written into the encoding and demodulation record, and the index of the transmission code sequence is saved.

5. The robust recognition method for infrared and visible light touch multi-source fusion as described in claim 4, characterized in that: Matched correlation demodulation samples the interaction region within the verification time window to form a received sampling sequence, performs zero-filling shift on the transmit code sequence, and calculates the correlation response sequence within a finite shift range. Based on the peak candidate window and sidelobe window, the correlation peak value, peak-side ratio, peak position consistency marker, and peak shape stability marker are obtained as correlation demodulation quality indicators.

6. The robust multi-source fusion recognition method for infrared and visible light touch as described in claim 5, characterized in that: The candidate touch area is generated centered on the fingertip and mapped to the infrared receiving screen. When the visible light side meets the condition that the fingertip enters the interaction area and the speed decreases and remains stable in a continuous frame, a verification time window of a one-time verification code sequence is triggered, and repeated triggering is suppressed within the same verification time window.

7. The robust multi-source fusion recognition method for infrared and visible light touch as described in claim 6, characterized in that: The one-time verification code sequence is generated by the key and timestamp through bitwise XOR, cyclic shift and byte permutation to generate an index. The corresponding transmission code sequence is selected from the candidate code sequence set and used for this verification. The receiving end determines the matching template according to the index and compares it with the relevant response sequence, and writes the index and timestamp into the touch verification record. The relevant response sequence is calculated only within the candidate touch area. When the relevant peak position falls into the peak candidate window and the peak-side ratio reaches the threshold associated with infrared reliability and scintillation component, and satisfies the geometric proximity relationship with the fingertip, a valid verification conclusion of the touch verification record is generated.

8. The robust multi-source fusion recognition method for infrared and visible light touch as described in claim 7, characterized in that: The relevant demodulation quality indicators are mapped to infrared verification confidence, and the candidate touch confidence is combined with visible light reliability to map to visible light touch confidence. The gating fusion rules are selected based on the infrared reliability and the touch events are output. When the infrared reliability decreases, the system enters a conservative mode and the confirmation conditions are improved.

9. The robust multi-source fusion recognition method for infrared and visible light touch as described in claim 8, characterized in that: The hysteresis state machine transitions through four states: hovering, candidate, contact, and release. Different continuous verification time window counting conditions are used for entering and exiting contact. A state identifier is maintained for the same candidate touch area. When multiple candidate touch areas overlap, the attribution of the touch event is determined based on the infrared verification confidence level and peak position consistency mark. The transmission parameters, reception exposure, and gain are updated based on the environmental infrared interference fingerprint and touch verification records. The updated values ​​are then projected onto a preset set of levels and written into the update buffer. This process takes effect at the boundaries of adjacent verification time windows and keeps the transmission code sequence and transmission parameters unchanged within the same verification time window.

10. The robust recognition method for infrared and visible light touch multi-source fusion as described in claim 9, characterized in that: The receiver is an infrared camera with external trigger exposure. The controller outputs an external trigger signal to the infrared camera at the chip rate, so that it performs short exposure sampling in each chip cycle to form a receiving sampling sequence, and reads it out in the pixel set corresponding to the candidate touch area, and makes the verification time window correspond to the code length and chip rate in the whole cycle. Before statistics, an avoidance mask is generated based on the hand and fingertip information on the visible light side and applied to the infrared received data. The median of the neighborhood is replaced by a constant outlier table, and an edge shielding band is set to remove stray reflection pixels. After performing morphological closing operations on the boundary of the avoidance mask, background statistical channels and candidate neighborhood statistical channels are formed respectively.

Images