Method and system for image tracking based on microphone level

Abstract

The application discloses a method and system for image tracking based on microphone level, wherein the method comprises collecting level data of at least two microphones, and performing time window aggregation on the same level at a preset detection frequency; determining a threshold interval corresponding to each channel according to an environment noise floor and channel sensitivity obtained by pre-calibration, and generating a level event when the aggregated level exceeds the threshold interval; writing the level event of each channel into an event queue, arbitrating according to a preset event type priority and channel role priority, and determining a target event; generating a control instruction according to a mapping relationship between the target event and a camera, converting the control instruction into a communication protocol format corresponding to the target camera through a protocol adaptation layer, and then issuing and executing the control instruction; receiving instruction execution feedback information from the camera, and updating a current view state according to the feedback result. The application realizes continuous and stable control of the current speaker picture in a multi-person audio and video conference scene.

Description

Technical Field

[0001] This invention relates to the field of image tracking technology, and in particular to a method and system for image tracking based on microphone levels. Background Technology

[0002] In multi-person audio and video conferencing scenarios, multiple microphones are typically used to capture the voice signals of the participants, and cameras track the corresponding speakers. However, existing technologies suffer from the following prominent problems:

[0003] First, microphones from different manufacturers vary in sensitivity, noise floor, and output characteristics, making it impossible for the system to determine which microphone is in a valid speaking state based on a unified standard, resulting in unstable sound source identification results. Second, when multiple microphones simultaneously generate level change events, the system lacks a unified priority arbitration mechanism, making it unable to accurately determine which microphone should trigger a camera view switch, easily leading to frequent image jumps or framing errors.

[0004] Furthermore, different camera models use different communication protocols and command formats. Existing systems typically rely on specific manufacturers or a single interface for control, making it difficult to be compatible with heterogeneous devices and increasing deployment and maintenance complexity. Moreover, most current solutions lack execution feedback and anomaly fallback mechanisms. When control commands are not executed correctly by the camera or feedback is lost, the system cannot automatically correct or degrade, resulting in image lag or loss. Summary of the Invention

[0005] The purpose of this application is to propose a method and system for image tracking based on microphone levels, so as to solve the technical problem of poor system recognition accuracy and control stability.

[0006] To address the aforementioned technical problems, this application provides a method for image tracking based on microphone levels, employing the following technical solution:

[0007] A method for image tracking based on microphone levels includes the following steps:

[0008] Collect the level data from at least two microphones, and aggregate the same level data within a time window at a preset detection frequency to obtain the aggregated level value;

[0009] Based on the pre-calibrated ambient noise floor and channel sensitivity, the threshold range corresponding to each channel is determined. When the aggregated level exceeds the threshold range, a level event containing channel identifier, event type and timestamp is generated.

[0010] The level events of each channel are written into the event queue, and arbitration is performed according to the preset event type priority and channel role priority to determine the target event;

[0011] Based on the mapping relationship between the target event and the camera, a control command is generated, and after being converted into the communication protocol format corresponding to the target camera by the protocol adaptation layer, it is sent out for execution.

[0012] The system receives instruction execution feedback information from the camera and updates the current viewpoint state based on the feedback results. If the feedback information is not obtained within a preset number of attempts, a degradation strategy is executed to switch to a preset screen.

[0013] In one possible implementation, before the step of acquiring the level data from at least two microphones and aggregating the same level data within a time window at a preset detection frequency to obtain the aggregated level value, the method further includes:

[0014] Silent samples from each channel are collected at the start of the meeting to calculate the ambient noise floor and dynamic fluctuation range.

[0015] During the acquisition process, the detection threshold is dynamically adjusted based on the real-time deviation of silent samples to adapt to changes in environmental noise and maintain consistency in the level determination of each channel.

[0016] In one possible implementation, the step of acquiring level data from at least two microphones and performing time-window aggregation on the same level at a preset detection frequency to obtain an aggregated level value includes, when performing time-window aggregation on the same level:

[0017] A moving average is performed within a short time window to filter out sudden jitters, and a trend gain coefficient is calculated within a long time window.

[0018] When the difference between the aggregation results of the short time window and the long time window exceeds the preset stability threshold, the current aggregation level value is determined after a delay to prevent false triggering caused by instantaneous volume peaks.

[0019] In one possible implementation, the step of determining the threshold range corresponding to each channel based on the pre-calibrated ambient noise floor and channel sensitivity, and generating a level event containing channel identifier, event type, and timestamp when the aggregated level exceeds the threshold range, specifically includes:

[0020] Calculate the slope, duration, and peak amplitude of the aggregation level change to generate a confidence score;

[0021] The confidence scores are weighted and averaged within adjacent time windows. When the confidence scores of two consecutive time windows both exceed the trigger threshold, the level event is confirmed as a valid event and written into the event queue.

[0022] In one possible implementation, the step of writing the level events of each channel into an event queue, arbitrating according to preset event type priority and channel role priority, and determining the target event specifically includes:

[0023] For each level event, an event score is calculated, which is determined by a weighted combination of event type priority, role priority, and level intensity.

[0024] When there are multiple level events with similar scores, compare their timestamp differences and select the latest event first;

[0025] After identifying the target event, a hold duration and a cooldown duration are set to control the locking and restoration of the camera's view.

[0026] In one possible implementation, the step of generating control commands based on the target event-camera mapping relationship, and converting the control commands into the communication protocol format corresponding to the target camera through a protocol adaptation layer before issuing and executing them, includes generating control commands based on the target event-camera mapping relationship as follows:

[0027] Retrieve the communication protocol and parameter template of the target camera from the protocol mapping table, fill in the preset bit number to generate a control message;

[0028] Before issuing commands, a validity check is performed. If the check result is abnormal, the control command is not issued and the error event is recorded to prevent abnormal commands from causing the camera's viewpoint to shift.

[0029] In one possible implementation, the step of receiving instruction execution feedback information from the camera, updating the current viewpoint state based on the feedback result, and executing a degradation strategy to switch to a preset screen when no feedback information is obtained within a preset number of times, includes the following steps when receiving instruction execution feedback information from the camera:

[0030] In the event of a timeout, the same control command is simultaneously sent to the backup control channel.

[0031] When the feedback results from the main channel and the backup channel are consistent, the viewpoint lock is confirmed to be successful; otherwise, the exponential backoff retry strategy is executed, and the wide-angle view is switched to a downgraded display after the retry fails.

[0032] To address the aforementioned technical problems, this application also provides a system for image tracking based on microphone levels, employing the following technical solution:

[0033] A system for image tracking based on microphone levels includes:

[0034] The acquisition module is used to acquire the level data of at least two microphones and aggregate the same level data within a time window at a preset detection frequency to obtain the aggregated level value.

[0035] The generation module is used to determine the threshold range corresponding to each channel based on the pre-calibrated ambient noise floor and channel sensitivity. When the aggregated level exceeds the threshold range, a level event containing channel identifier, event type and timestamp is generated.

[0036] The arbitration module is used to write the level events of each channel into the event queue, arbitrate according to the preset event type priority and channel role priority, and determine the target event;

[0037] The conversion module is used to generate control commands based on the mapping relationship between the target event and the camera, and convert the control commands into the communication protocol format corresponding to the target camera through the protocol adaptation layer before sending them out for execution;

[0038] The receiving module is used to receive instruction execution feedback information from the camera, update the current view state according to the feedback result, and execute a degradation strategy to switch to a preset screen if the feedback information is not obtained within a preset number of times.

[0039] To address the aforementioned technical problems, this application also provides a computer device that employs the following technical solution:

[0040] A computer device includes a memory and a processor, the memory storing computer-readable instructions, the processor executing the computer-readable instructions to implement the steps of the method for image tracking based on microphone levels as described above.

[0041] To address the aforementioned technical problems, this application also provides a computer-readable storage medium, employing the technical solution described below:

[0042] A computer-readable storage medium storing computer-readable instructions that, when executed by a processor, implement the steps of the method for image tracking based on microphone levels as described above.

[0043] Compared with the prior art, the embodiments of this application have the following main advantages:

[0044] The image tracking method based on microphone levels disclosed in this application generates a unified level event by aggregating the level data of at least two microphones over a time window and combining it with a threshold range determined by pre-calibrated ambient noise floor and channel sensitivity. The level event is written into an event queue and arbitrated according to event type priority and channel role priority to determine the target event. Based on the mapping relationship between the target event and the camera, the control command is converted into the communication protocol format corresponding to the target camera through a protocol adaptation layer and then sent out. The method also receives instruction execution feedback information from the camera and executes a degradation strategy when no feedback is received. This achieves continuous, stable, heterogeneous device compatible, and closed-loop verifiable automatic tracking and unified viewpoint control of the current speaker's image in a multi-person audio and video conferencing scenario. Attached Figure Description

[0045] To more clearly illustrate the solutions in this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is a flowchart of an embodiment of the image tracking method based on microphone level according to this application;

[0047] Figure 2 This is a schematic diagram of an embodiment of a system for image tracking based on microphone levels according to this application;

[0048] Figure 3 This is a schematic diagram of the structure of one embodiment of the computer device according to this application. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0050] refer to Figure 1 The diagram illustrates a flowchart of an embodiment of a microphone-level-based image tracking method according to this application. The microphone-level-based image tracking method includes the following steps:

[0051] Step S101: Collect the level data of at least two microphones, and aggregate the same level data within a time window at a preset detection frequency to obtain the aggregated level value.

[0052] In this embodiment, the electronic device on which the image tracking method based on microphone levels operates can send or receive data via a wired or wireless connection. It should be noted that the aforementioned wireless connection methods may include, but are not limited to, 3G / 4G / 5G connections, WiFi connections, Bluetooth connections, WiMAX connections, Zigbee connections, UWB (ultrawideband) connections, and other currently known or future-developed wireless connection methods.

[0053] In this embodiment, the system first acquires level data from at least two microphone interfaces at a fixed sampling and detection rhythm. The detection frequency can be set between 20 and 100 Hz according to room reverberation and speaking speed. To suppress transient fluctuations and preserve speaking trends, the same channel's level is aggregated within a sliding time window. A common approach is to calculate the mean or median of the most recent K samples, and if necessary, add quantile or exponential weighting to improve robustness, thereby obtaining the aggregated level value used for judgment. For example, at a detection frequency of 50 Hz, a short window of 200 ms can be selected in conjunction with a long window of 800 ms, with each output serving as a component of the aggregated level. The short window is used for fast response, and the long window is used to suppress occasional pulses. When a single channel experiences clipping or frame loss, it is repaired by interpolation of effective samples from adjacent time windows to ensure the continuous usability of the aggregated level.

[0054] Step S102: Determine the threshold range corresponding to each channel based on the pre-calibrated ambient noise floor and channel sensitivity. When the aggregated level exceeds the threshold range, generate a level event containing channel identifier, event type and timestamp.

[0055] In this embodiment, to ensure comparability of microphones of different brands and sensitivities under the same judgment standard, the system performs pre-calibration during the conference startup or device online phase. It silently collects several seconds of level data for each channel to estimate the ambient noise floor, and combines this with the device specification sheet or a one-time reading sample to estimate the channel sensitivity, thereby determining the threshold range and hysteresis bandwidth for each channel. During operation, when the aggregated level of a channel exceeds the upper threshold of that channel and continues to reach the minimum trigger duration (e.g., ≥120 ms), it is determined as a speech trigger, and a level event is generated accordingly. When the level falls back below the lower threshold and continues for a certain period (e.g., ≥200 ms), it is determined as a speech end event. The event object includes at least the channel identifier, event type (microphone on, microphone off, or microphone audio input), and timestamp. To reduce false triggers, a confidence score may be included in the embodiment. This score is calculated by weighting the aggregated level amplitude, rising edge slope, and duration, and is used for subsequent filtering but does not change the main judgment logic. For example, under the conditions of a noise floor of 32 dB and a sensitivity of S=1.2, the threshold range of channel CH3 can be set to 42-38 dB (including hysteresis band). When the aggregated level stably exceeds 42 dB and lasts for 150 ms, the CH3_audio input_ event is generated.

[0056] Step S103: Write the level events of each channel into the event queue, and arbitrate according to the preset event type priority and channel role priority to determine the target event.

[0057] In this embodiment, the system writes the level events generated by each channel into the event queue and performs priority arbitration. Arbitration adopts a unified, configurable criterion: first, it compares the priority of event type (e.g., audio input events are higher than on / off events), then it compares the priority of channel role (e.g., the host's microphone is higher than ordinary and audience microphones). If they are still tied, it compares the level intensity or timestamp, prioritizing the newest one with a higher amplitude. To prevent screen jitter, once the target event output by arbitration is established, it enters a hold period. During the hold period, preemption is only allowed if the comparison result of a new event significantly exceeds the current target (e.g., the amplitude difference exceeds a set threshold or comes from a higher role). After the hold period ends, a cooldown period begins, during which low-intensity or short-duration events are not switched. Taking the concurrent operation of conference host CH1 and speaker CH3 as an example, if the rule is set to "host role priority, but strong preemption is allowed," when the amplitude and duration of CH3's audio input event are significantly higher than the threshold and exceed the preemption difference, the view can be switched from the host's view to the speaker's view during the hold period, ensuring both stable framing and responsiveness to actual speaking.

[0058] Step S104: Generate control commands based on the mapping relationship between the target event and the camera, and then convert the control commands into the communication protocol format corresponding to the target camera through the protocol adaptation layer before sending them out for execution.

[0059] In this embodiment, once the target event is determined, the system generates control commands based on the mapping relationship between the target event and the camera. The mapping relationship can be provided during the installation and debugging phase, such as "CH3 → Camera CAM2 preset bit P05". The control commands are first described in a unified internal format (text or binary structure), then converted by the protocol adaptation layer into communication protocol frames supported by the target camera and sent for execution. The protocol adaptation layer masks differences, supporting both serial interfaces (RS-232 / RS-485) and network interfaces based on TCP or UDP; it parameterizes command templates, baud rates, parity bits, network ports, and authentication methods from different manufacturers, thus ensuring compatibility with heterogeneous devices without altering the upper-layer logic. To avoid frequent mechanical movement or network congestion, the system merges consecutive same-direction control requests within a 100-300 ms window, retaining only the latest valid preset bit call; when the queue detects that the same camera repeatedly calls the same preset bit, the adaptation layer performs idempotent processing and does not send the command repeatedly.

[0060] Step S105: Receive instruction execution feedback information from the camera, update the current viewpoint state according to the feedback result, and if the feedback information is not obtained within a preset number of times, execute a degradation strategy to switch to a preset screen.

[0061] In this embodiment, after control is completed, the system enters the execution feedback phase. The system receives instruction execution feedback information from the camera. The feedback source can be any one or more of the following: serial port response frames, network confirmation messages, or media layer to stream confirmation: for example, a serial port returning an "OK" code, a JSON response on the TCP connection, or the media service detecting changes in the image stream and timestamp corresponding to the target camera's view. Upon receiving valid feedback, the current view state is updated and the relevant retry count is cleared. If no feedback information is received within a preset number of attempts (e.g., three retries, total wait of 2 seconds), the control is deemed to have failed, triggering a degradation strategy to switch to a preset view, such as a panoramic wide-angle view or the preset position with the most recent valid confirmation, to ensure that the meeting room view is not interrupted. At the same time, the reason for failure and context (target event, mapping relationship, sending channel, return code, and time) are recorded for subsequent maintenance troubleshooting or adaptive adjustment. The above feedback and degradation design enables the control link to have closed-loop and convergence capabilities: even if equipment failure or network anomalies occur, the system can recover to a safe view within a foreseeable time limit, avoiding the impact of black screens, incorrect images, or frequent flashing on the meeting experience.

[0062] This application achieves continuous, stable, heterogeneous device compatible, and closed-loop verifiable automatic tracking and unified viewpoint control of the current speaker's image in multi-person audio and video conferencing scenarios by aggregating the level data of at least two microphones within a time window and combining the pre-calibrated ambient noise floor with the threshold range determined by the channel sensitivity. The level events are then written into an event queue and arbitrated according to event type priority and channel role priority to determine the target event. Based on the mapping relationship between the target event and the camera, the control command is converted into the communication protocol format corresponding to the target camera via a protocol adaptation layer and then sent out. Furthermore, the application receives instruction execution feedback information from the camera and executes a degradation strategy when no feedback is received.

[0063] In some optional implementations of this embodiment, before the step of acquiring the level data of at least two microphones and aggregating the same level data within a time window at a preset detection frequency to obtain the aggregated level value, the method further includes:

[0064] Silent samples from each channel are collected at the start of the meeting to calculate the ambient noise floor and dynamic fluctuation range.

[0065] During the acquisition process, the detection threshold is dynamically adjusted based on the real-time deviation of silent samples to adapt to changes in environmental noise and maintain consistency in the level determination of each channel.

[0066] In this embodiment, a pre-adaptive acquisition process is implemented. This involves acquiring silent samples from each channel at the start of the meeting to calculate the ambient noise floor and dynamic fluctuation range. During the actual acquisition process, the detection threshold is dynamically adjusted based on the real-time deviation of the silent samples. This is because the noise floor and fluctuation range vary between different meeting rooms and different channels. Using a uniform threshold directly would lead to some channels being consistently "too tight" or "too loose." In practice, silent sample acquisition is automatically triggered during the pre-meeting countdown, lasting several seconds to over ten seconds to estimate the ambient noise floor and fluctuation range for each channel. During operation, if the statistics of the silent segment deviate from the baseline at startup, the detection threshold for that channel is fine-tuned according to the deviation, ensuring the threshold always matches the current environment. For example, turning on the air conditioner or changing the number of attendees alters the room noise floor. If the median value of the silent sample for channel CH2 rises by 2 dB from the baseline, the upper and lower edges of the threshold range for that channel are raised by the same amount or by a preset coefficient, thus maintaining consistency and comparability in the level determination of different channels. The output of this preprocessing step directly serves the determination of subsequent threshold ranges and the stable generation of level events, enabling the main link to have environmental adaptability from the source.

[0067] This application collects silent samples and calculates the ambient noise floor and dynamic fluctuation range when the meeting starts. At the same time, it dynamically corrects the detection threshold based on the real-time deviation of the silent samples during the collection process, so that the system can adapt to different ambient noise and channel sensitivity changes, thereby maintaining the consistency and stability of multi-channel level judgment.

[0068] In some optional implementations of this embodiment, the step of acquiring level data from at least two microphones and performing time-window aggregation on the same level at a preset detection frequency to obtain an aggregated level value includes:

[0069] A moving average is performed within a short time window to filter out sudden jitters, and a trend gain coefficient is calculated within a long time window.

[0070] When the difference between the aggregation results of the short time window and the long time window exceeds the preset stability threshold, the current aggregation level value is determined after a delay to prevent false triggering caused by instantaneous volume peaks.

[0071] In this embodiment, a moving average is performed within a short time window to filter out sudden jitter, and a trend gain coefficient is calculated within a long time window. A delay judgment is initiated when the difference between the aggregated results of the short and long windows exceeds a preset stability threshold. The key point is that the short window is responsible for quickly tracking the start of speech, while the long window provides steady-state and trend references. The difference between the two is equivalent to a "stability indicator." When the difference is too large, it often indicates a momentary peak such as tapping on the table or flipping a piece of paper, rather than actual speech. Delay judgment can prevent false triggering in this case. In practice, the short time window can be a moving average or median filter of 100–300 ms, and the long time window can be 600–1200 ms, with a trend gain coefficient calculated to suppress isolated peaks. The empirical value of the stability threshold can be set according to the system's quantization unit. For example, when the difference between the short and long windows exceeds 3 dB, a delay observation period of several hundred milliseconds is entered. If the difference quickly falls back, the aggregated level value is not updated; if the difference converges, the current aggregated level value is confirmed to be effective. In real-world scenarios, a teacup lightly touching the table will create a spike in the short window, while the long window will not rise accordingly due to inertia. The difference between the two will increase briefly, and the delayed judgment will effectively prevent false triggering. However, a real speech will raise both the short and long windows simultaneously within hundreds of milliseconds, with the difference being small and converging quickly, allowing the system to pass through without being noticed.

[0072] This application enables the system to distinguish between real speech and transient noise events by performing a moving average within a short time window to filter out sudden jitter, calculating the trend gain coefficient within a long time window, and delaying the determination of the aggregated level value when the difference between the short and long window results exceeds the stability threshold. This effectively reduces the false trigger rate and improves the accuracy and robustness of speech detection.

[0073] In some optional implementations of this embodiment, the step of determining the threshold range corresponding to each channel based on the pre-calibrated ambient noise floor and channel sensitivity, and generating a level event containing channel identifier, event type, and timestamp when the aggregated level exceeds the threshold range, specifically includes:

[0074] Calculate the slope, duration, and peak amplitude of the aggregation level change to generate a confidence score;

[0075] The confidence scores are weighted and averaged within adjacent time windows. When the confidence scores of two consecutive time windows both exceed the trigger threshold, the level event is confirmed as a valid event and written into the event queue.

[0076] In this embodiment, during event generation, the slope of the aggregated level change, duration, and peak amplitude are calculated to form a confidence score. The confidence score is then weighted and averaged within adjacent time windows. Only when the confidence score of two consecutive time windows exceeds the trigger threshold is the level event confirmed as valid and written into the event queue. The core function is to elevate the "threshold exceeding" condition from a single amplitude condition to a composite condition of amplitude and dynamic shape, thereby significantly reducing false events caused by occasional noise or short peaks. In implementation, the slope can be taken as the first-order difference of the aggregated level, the duration is calculated based on the cumulative time across windows, and the peak amplitude is the maximum aggregated level within the current window. These three factors are weighted according to experience or training to synthesize a confidence score, which is then weighted and averaged across adjacent time windows to smooth out jitter. For example, if the confidence levels of channel CH3 in two adjacent windows are 0.76 and 0.81 respectively, both exceeding the preset trigger threshold of 0.7, then a level event of type microphone audio input will be generated immediately; if the first window is 0.75 and the second window drops to 0.55, then no confirmation will be made, effectively suppressing short-lived interference.

[0077] This application generates a confidence score by calculating the slope of the aggregated level change, the duration, and the peak amplitude, and performs a weighted average of the confidence scores within adjacent time windows. The level event is confirmed as valid only when the confidence scores of consecutive time windows all exceed the threshold. This enables the system to comprehensively judge the strength and duration of the spoken signal, thereby improving the reliability of event generation and reducing false triggers.

[0078] In some optional implementations of this embodiment, the steps of writing the level events of each channel into the event queue, arbitrating according to the preset event type priority and channel role priority, and determining the target event specifically include:

[0079] For each level event, an event score is calculated, which is determined by a weighted combination of event type priority, role priority, and level intensity.

[0080] When there are multiple level events with similar scores, compare their timestamp differences and select the latest event first;

[0081] After identifying the target event, a hold duration and a cooldown duration are set to control the locking and restoration of the camera's view.

[0082] In this embodiment, the multi-factor decision-making process calculates an event score for each level event. The event score is determined by a weighted combination of event type priority, role priority, and level intensity. When multiple level events with similar scores exist, the latest event is selected based on the timestamp difference. After the target event is determined, a hold duration and a cooldown duration are set to control the locking and unlocking of the camera's view. This arbitration mechanism has three levels: type priority ensures that "speaking events" precede "switching events"; role priority ensures that "key roles such as hosts" have the right to speak when needed; level intensity provides an objective instantaneous comparison; and when scores are close, adjudicating based on time freshness avoids interference from long-tail events. The hold duration prevents the screen from swinging back and forth between events with similar scores, and the cooldown duration provides a "stabilization window" for the recently concluded target. In a scenario where the host and speaker are speaking concurrently, if the level intensity and duration of the speaker's channel significantly increase, the event score surpasses that of the host's channel, and the timestamp is updated, the system switches to the speaker's view while adhering to the policy, ensuring the priority of authoritative roles while allowing for reasonable preemption when speaking.

[0083] This application calculates a weighted score for each level event, consisting of event type priority, role priority, and level intensity. When the scores are close, the timestamps are compared to determine the latest event. At the same time, a hold duration and a cooldown duration are set to achieve orderly arbitration of concurrent events and smooth control of perspective switching, thereby maintaining the stability of the screen and the rationality of the response in the case of multiple people speaking.

[0084] In some optional implementations of this embodiment, the step of generating control commands based on the target event and camera mapping relationship, and converting the control commands into the communication protocol format corresponding to the target camera through a protocol adaptation layer before issuing and executing them, includes the following steps when generating control commands based on the target event and camera mapping relationship:

[0085] Retrieve the communication protocol and parameter template of the target camera from the protocol mapping table, fill in the preset bit number to generate a control message;

[0086] Before issuing commands, a validity check is performed. If the check result is abnormal, the control command is not issued and the error event is recorded to prevent abnormal commands from causing the camera's viewpoint to shift.

[0087] In this embodiment, when generating control commands, the communication protocol and parameter template of the target camera need to be retrieved from the protocol mapping table. Preset bit numbers are then filled in to generate control messages. Before sending, command validity is checked; if the check fails, the command is not sent and the error event is recorded to prevent abnormal commands from causing camera viewpoint shifts. This embodiment standardizes and secures the path from target events to control messages: the protocol mapping table uniformly maintains command formats, parameter positions, verification methods, baud rates, or port numbers for different manufacturers under serial interfaces or TCP / UDP network interfaces; the parameter template ensures that fields such as preset bit numbers and lens speeds are filled in according to regulations; command validity checks are used to detect out-of-bounds preset bits, illegal parameter combinations, or field-state conflicts. Once detected, the command is intercepted and logged. In practice, if the mapping relationship requires calling CAM2's P05, and the template indicates that the preset bit range for this model is P01 to P20, then filling in P05 and passing it through CRC or field verification is sufficient; if a maintenance error results in the generation of P25, it will be intercepted before sending to prevent lens misalignment or device errors.

[0088] This application enables the system to generate and securely issue control commands in a unified manner across multiple brands and heterogeneous protocols of cameras by retrieving communication protocols and parameter templates from the protocol mapping table, filling in preset bit numbers, and performing command validity checks when generating control commands. This ensures the compatibility and reliability of cross-device control.

[0089] In some optional implementations of this embodiment, the step of receiving instruction execution feedback information from the camera, updating the current viewpoint state based on the feedback result, and executing a degradation strategy to switch to a preset screen when no feedback information is obtained within a preset number of times, includes receiving instruction execution feedback information from the camera as follows:

[0090] In the event of a timeout, the same control command is simultaneously sent to the backup control channel.

[0091] When the feedback results from the main channel and the backup channel are consistent, the viewpoint lock is confirmed to be successful; otherwise, the exponential backoff retry strategy is executed, and the wide-angle view is switched to a downgraded display after the retry fails.

[0092] In this embodiment, a backup control channel and an exponential backoff retry strategy are introduced. In the event of a timeout, the same control command is simultaneously sent to the backup control channel. When the feedback results from the primary and backup channels are consistent, the viewpoint lock is confirmed to be successful; otherwise, a retry is performed according to the exponential backoff strategy. If the retry ultimately fails, the display switches to a wide-angle view for degraded display. This embodiment reduces the risk of "single-channel unreliability" to a reliable closed loop of "multi-channel consistency confirmation." It can provide backup through a serial or UDP link during brief fluctuations in the primary TCP link, and it can also prevent conflicting feedback from two channels from causing false successes through consistency checks. Exponential backoff is used to increase the retry interval when the network or equipment is busy, preventing cascading failures; degrading to a wide-angle view ensures continuous visibility of the scene, preventing black screens or displaying in incorrect preset positions. In practice, if the main channel times out while the backup channel returns a valid response, the system immediately updates the current view status with the backup result. If the return codes of the two channels are inconsistent or both time out, the system waits for retries in increments of 1, 2, and 4 times. If the system still fails, it will perform a downgrade to ensure that the end user always receives meaningful screen output.

[0093] This application enables the system to have multi-channel redundancy and self-healing capabilities by simultaneously sending the same command to the backup control channel in the event of a feedback timeout, and confirming success when the results of the primary and backup channels are consistent; otherwise, it retryes according to an exponential backoff strategy until it is downgraded to a wide-angle view. This allows the system to maintain the continuity of the image output and the recoverability of the system control in the event of communication abnormalities or equipment failures.

[0094] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing related hardware through computer-readable instructions. These computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, optical disk, or read-only memory (ROM), or random access memory (RAM).

[0095] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0096] Further reference Figure 2 As a response to the above Figure 1 The implementation of the method shown in this application provides an embodiment of a system for image tracking based on microphone levels. This system embodiment is similar to... Figure 1 Corresponding to the method embodiments shown, the system can be specifically applied to various electronic devices.

[0097] like Figure 2 As shown, the image tracking system 200 based on microphone level described in this embodiment includes: an acquisition module 201, a generation module 202, an arbitration module 203, a conversion module 204, and a receiving module 205. Wherein:

[0098] The acquisition module 201 is used to acquire the level data of at least two microphones and aggregate the same level data within a time window at a preset detection frequency to obtain the aggregated level value.

[0099] The generation module 202 is used to determine the threshold range corresponding to each channel based on the pre-calibrated ambient noise floor and channel sensitivity. When the aggregated level exceeds the threshold range, a level event containing channel identifier, event type and timestamp is generated.

[0100] Arbitration module 203 is used to write the level events of each channel into the event queue, arbitrate according to the preset event type priority and channel role priority, and determine the target event;

[0101] The conversion module 204 is used to generate control commands based on the mapping relationship between the target event and the camera, and convert the control commands into the communication protocol format corresponding to the target camera through the protocol adaptation layer before sending them out for execution.

[0102] The receiving module 205 is used to receive instruction execution feedback information from the camera, update the current view state according to the feedback result, and execute a degradation strategy to switch to a preset screen if the feedback information is not obtained within a preset number of times.

[0103] The system for image tracking based on microphone level provided in this embodiment of the invention can implement all the processes of the image tracking method based on microphone level in the above embodiments. The functions and technical effects of each module in the device are the same as those of the image tracking method based on microphone level in the above embodiments, and will not be repeated here.

[0104] To address the aforementioned technical problems, embodiments of this application also provide a computer device. Please refer to [link / reference needed]. Figure 3 , Figure 3 This is a basic structural block diagram of the computer device in this embodiment.

[0105] The computer device 3 includes a memory 31, a processor 32, and a network interface 33 that are interconnected via a system bus. It should be noted that only the computer device 3 with components 31-33 is shown in the figure; however, it should be understood that it is not required to implement all the shown components, and more or fewer components can be implemented alternatively. Those skilled in the art will understand that the computer device described here is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0106] The computer device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device can interact with the user via a keyboard, mouse, remote control, touchpad, or voice control.

[0107] The memory 31 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 31 may be an internal storage unit of the computer device 3, such as the hard disk or memory of the computer device 3. In other embodiments, the memory 31 may also be an external storage device of the computer device 3, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 3. Of course, the memory 31 may also include both the internal storage unit and its external storage device of the computer device 3. In this embodiment, the memory 31 is typically used to store the operating system and various application software installed on the computer device 3, such as computer-readable instructions for image tracking methods based on microphone levels. In addition, the memory 31 can also be used to temporarily store various types of data that have been output or will be output.

[0108] In some embodiments, the processor 32 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor 32 is typically used to control the overall operation of the computer device 3. In this embodiment, the processor 32 is used to execute computer-readable instructions stored in the memory 31 or to process data, for example, to execute computer-readable instructions for the method of image tracking based on microphone levels.

[0109] The network interface 33 may include a wireless network interface or a wired network interface, which is typically used to establish communication connections between the computer device 3 and other electronic devices.

[0110] The computer device provided in this application generates a unified level event by aggregating the level data of at least two microphones within a time window and combining it with the threshold range determined by the pre-calibrated ambient noise floor and channel sensitivity. The level event is written into an event queue and arbitrated according to event type priority and channel role priority to determine the target event. Based on the mapping relationship between the target event and the camera, the control command is converted into the communication protocol format corresponding to the target camera via a protocol adaptation layer and then sent out. The device also receives instruction execution feedback information from the camera and executes a degradation strategy when no feedback is received. This achieves continuous, stable, heterogeneous device-compatible, and closed-loop-verifiable automatic tracking and unified viewpoint control of the current speaker's image in a multi-person audio and video conferencing scenario.

[0111] This application also provides another embodiment, namely, providing a computer-readable storage medium storing computer-readable instructions that can be executed by at least one processor to cause the at least one processor to perform the steps of the above-described method for image tracking based on microphone levels.

[0112] The computer-readable storage medium provided in this application generates a unified level event by aggregating the level data of at least two microphones within a time window and combining it with the threshold range determined by the pre-calibrated ambient noise floor and channel sensitivity. The level event is written into an event queue and arbitrated according to event type priority and channel role priority to determine the target event. Based on the mapping relationship between the target event and the camera, the control command is converted into the communication protocol format corresponding to the target camera via a protocol adaptation layer and then sent out. The medium also receives instruction execution feedback information from the camera and executes a degradation strategy when no feedback is received. This enables continuous, stable, heterogeneous device-compatible, and closed-loop-verifiable automatic tracking and unified viewpoint control of the current speaker's image in a multi-person audio and video conferencing scenario.

[0113] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0114] The above are merely preferred embodiments of this application and are not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for image tracking based on microphone levels, characterized in that, Includes the following steps: Collect the level data from at least two microphones, and aggregate the same level data within a time window at a preset detection frequency to obtain the aggregated level value; Based on the pre-calibrated ambient noise floor and channel sensitivity, the threshold range corresponding to each channel is determined. When the aggregated level exceeds the threshold range, a level event containing channel identifier, event type and timestamp is generated. The level events of each channel are written into the event queue, and arbitration is performed according to the preset event type priority and channel role priority to determine the target event; Based on the mapping relationship between the target event and the camera, a control command is generated, and after being converted into the communication protocol format corresponding to the target camera by the protocol adaptation layer, it is sent out for execution. The system receives instruction execution feedback information from the camera and updates the current viewpoint state based on the feedback results. If the feedback information is not obtained within a preset number of attempts, a degradation strategy is executed to switch to a preset screen.

2. The image tracking method based on microphone level according to claim 1, characterized in that, Before the step of acquiring the level data from at least two microphones and aggregating the same level data within a time window at a preset detection frequency to obtain the aggregated level value, the method further includes: Silent samples from each channel are collected at the start of the meeting to calculate the ambient noise floor and dynamic fluctuation range. During the acquisition process, the detection threshold is dynamically adjusted based on the real-time deviation of silent samples to adapt to changes in environmental noise and maintain consistency in the level determination of each channel.

3. The image tracking method based on microphone level according to claim 2, characterized in that, In the step of acquiring level data from at least two microphones and performing time-window aggregation on the same level at a preset detection frequency to obtain an aggregated level value, the time-window aggregation on the same level includes: A moving average is performed within a short time window to filter out sudden jitters, and a trend gain coefficient is calculated within a long time window. When the difference between the aggregation results of the short time window and the long time window exceeds the preset stability threshold, the current aggregation level value is determined after a delay to prevent false triggering caused by instantaneous volume peaks.

4. The method for image tracking based on microphone level according to claim 1, characterized in that, The step of determining the threshold range corresponding to each channel based on the pre-calibrated environmental noise floor and channel sensitivity, and generating a level event containing channel identifier, event type, and timestamp when the aggregated level exceeds the threshold range, specifically includes: Calculate the slope, duration, and peak amplitude of the aggregation level change to generate a confidence score; The confidence scores are weighted and averaged within adjacent time windows. When the confidence scores of two consecutive time windows both exceed the trigger threshold, the level event is confirmed as a valid event and written into the event queue.

5. The image tracking method based on microphone level according to claim 1, characterized in that, The steps of writing the level events of each channel into the event queue, arbitrating according to the preset event type priority and channel role priority, and determining the target event specifically include: For each level event, an event score is calculated, which is determined by a weighted combination of event type priority, role priority, and level intensity. When there are multiple level events with similar scores, compare their timestamp differences and select the latest event first; After identifying the target event, a hold duration and a cooldown duration are set to control the locking and restoration of the camera's view.

6. The method for image tracking based on microphone level according to claim 1, characterized in that, In the step of generating control commands based on the target event-camera mapping relationship, and converting the control commands into the communication protocol format corresponding to the target camera through a protocol adaptation layer before issuing and executing them, generating control commands based on the target event-camera mapping relationship includes: Retrieve the communication protocol and parameter template of the target camera from the protocol mapping table, fill in the preset bit number to generate a control message; Before issuing commands, a validity check is performed. If the check result is abnormal, the control command is not issued and the error event is recorded to prevent abnormal commands from causing the camera's viewpoint to shift.

7. The method for image tracking based on microphone level according to claim 1, characterized in that, In the step of receiving instruction execution feedback information from the camera, updating the current viewpoint state based on the feedback result, and executing a degradation strategy to switch to a preset screen when no feedback information is obtained within a preset number of times, receiving instruction execution feedback information from the camera includes: In the event of a timeout, the same control command is simultaneously sent to the backup control channel. When the feedback results from the main channel and the backup channel are consistent, the viewpoint lock is confirmed to be successful; otherwise, the exponential backoff retry strategy is executed, and the wide-angle view is switched to a downgraded display after the retry fails.

8. A system for image tracking based on microphone levels, characterized in that, include: The acquisition module is used to acquire the level data of at least two microphones and aggregate the same level data within a time window at a preset detection frequency to obtain the aggregated level value. The generation module is used to determine the threshold range corresponding to each channel based on the pre-calibrated ambient noise floor and channel sensitivity. When the aggregated level exceeds the threshold range, a level event containing channel identifier, event type and timestamp is generated. The arbitration module is used to write the level events of each channel into the event queue, arbitrate according to the preset event type priority and channel role priority, and determine the target event; The conversion module is used to generate control commands based on the mapping relationship between the target event and the camera, and convert the control commands into the communication protocol format corresponding to the target camera through the protocol adaptation layer before sending them out for execution; The receiving module is used to receive instruction execution feedback information from the camera, update the current view state according to the feedback result, and execute a degradation strategy to switch to a preset screen if the feedback information is not obtained within a preset number of times.

9. A computer device, characterized in that, The method includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor executes the computer-readable instructions to implement the steps of the method for image tracking based on microphone level as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-readable instructions that, when executed by a processor, implement the steps of the method for image tracking based on microphone levels as described in any one of claims 1 to 7.

Images