Conference Room Audio and Video Control Method Based on Artificial Intelligence and Audio-Video Linkage

Abstract

This invention proposes a conference room audio and video control method based on artificial intelligence and audio-visual linkage orchestration. To solve the problems of misjudgment during speaker switching, audio-visual asynchrony, and unstable cross-device collaboration in existing methods, the method proposes: acquiring the node capabilities of cameras, microphones, and displays, as well as the spatial status of participants, and establishing a unified time base and audio-visual linkage channel; generating candidate speaker objects and event markers based on aligned session events; constructing a semantic model containing rhythmic preamble, momentum judgment, and displacement memory, and outputting control signals, executable windows, and pre-configured amplitudes; and integrating camera pre-positioning trajectories, microphone secondary acquisition channels, display layout, and takeover order within the executable window, calculating rhythm alignment points, and forming a switching execution list to drive the linkage of camera PTZ, microphone, and display. This method ensures audibility and smooth switching during transitions and is suitable for audio-visual acquisition and display linkage scenarios with multiple participants.

Description

Technical Field

[0001] The present invention belongs to the technical field of audio - video linkage control, and particularly relates to a method for controlling audio - video in a conference room based on artificial intelligence and audio - video linkage compilation and arrangement. Background Art

[0002] In the scenarios of meetings and seminars with multiple participants, cameras PTZ, microphones, and display terminals need to cooperate to complete audio - video collection and presentation. With the increase in multi - person speaking, relay speaking, short interjections, and overlapping conversations, how to complete the timely identification and transition of the main speaker object without interrupting the viewing and listening experience has become an important basis for audio - video linkage control. This requires unified modeling and coordinated control of device capabilities, the positions and orientations of participants, and conversation rhythms.

[0003] Existing technologies usually include: collecting device capabilities such as camera preset positions, microphone pointing and channels, and display layouts, establishing a spatial mapping in combination with the positions and orientations of participants; aligning conversation events based on a unified time base, determining candidate main speaker objects using features such as speech start and end, energy, mouth - opening, and line - of - sight pointing, and screening and sorting them according to thresholds and durations; on this basis, controlling and arranging camera positioning, audio collection, and display switching according to device response characteristics and preset positions, and performing switching and display in combination with certain delay compensation and simple timing rules.

[0004] Existing solutions generally lack the definition of an executable window with pause and connection time slots as the core and the fusion determination of entry trends. It is difficult to perform integrated arrangement and rhythm alignment of camera pre - positions, microphone secondary collection, and display layouts across devices under a unified time base, resulting in problems such as false triggering, audio - video out - of - sync, and uneven transitions in scenarios such as interjections and takeovers.

[0005] Therefore, how to provide a method for controlling audio - video in a conference room that can solve the above - mentioned technical problems is an issue that needs to be addressed by those skilled in the art. Summary of the Invention

[0006] An object of the present invention is to provide a method for controlling audio - video in a conference room based on artificial intelligence and audio - video linkage compilation and arrangement. In a multi - person conversation, based on a unified time base and device response delay constraints, an executable window is generated within pause and connection time slots using conversation rhythms and multi - modal states, and accordingly, linkage compilation and rhythm alignment are performed on camera pre - positions, microphone secondary collection, and display layouts to achieve smooth transition of the main speaker object and ensure audibility.

[0007] A method for controlling audio - video in a conference room based on artificial intelligence and audio - video linkage compilation and arrangement according to an embodiment of the present invention includes:

[0008] S1. Obtain audio and video nodes and participants, establish an audio and video linkage channel and set the main speaker, and output the status of the audio and video linkage channel.

[0009] S2. Receive session events based on the status of the audio-visual linkage channel, determine the candidate speaker, and output the status and event flags of the candidate speaker.

[0010] S3. Based on the candidate speaker's status and event markers, construct a conversation transition intention prediction model. The conversation transition intention prediction model includes a rhythm preamble unit, a momentum determination unit, and a displacement memory unit. The rhythm preamble unit determines the pause time slots, the momentum determination unit combines line of sight and lip movement to identify the speaker's posture, the displacement memory unit estimates the entry trend, generates control signals, executable windows, and pre-configured amplitudes, and outputs timing results.

[0011] S4. Based on the timing results, a linkage transition plan is generated using a linkage arrangement framework controlled by the main speaker object: the camera pre-position trajectory is formed according to the pre-allocation amplitude, the microphone secondary acquisition channel is configured, the display layout and the main acquisition takeover order are arranged within the executable window, and the actions of the camera, microphone and display are constrained by the executable window, and the linkage transition plan is output.

[0012] S5. Calculate the rhythm alignment point and output the switching execution list based on the linkage transition plan and executable window;

[0013] S6. Determine the target execution list based on control signals;

[0014] S7. Generate node control instructions based on the target execution list to drive the linkage of camera PTZ, microphone main acquisition and display layout, and output the status of the main subject.

[0015] Optionally, S1 is as follows:

[0016] Acquire preset position information and reachable angle range of camera PTZ, acquisition direction and switchable channels of microphone, available display layout and switching mode, and generate audio and video node information;

[0017] Collect the location and orientation of the participants, obtain the standing area and movement status of the participants, establish the spatial mapping relationship between the participants and audio and video nodes, and generate participant information;

[0018] An audio-visual linkage channel is established by using audio and video node information and participant information, a unified time base and control path are configured, and the response latency and action execution boundary of the audio-visual linkage channel are measured to form an audio-visual linkage channel.

[0019] Set a speaking duration threshold and a position stability threshold, and set the participants who meet the speaking duration threshold and position stability as the main speakers;

[0020] The audio-visual linkage channel, audio-visual node information, participant information, and speaker information are summarized to output the audio-visual linkage channel status. The audio-visual linkage channel status includes a unified time base, control path, response delay, spatial mapping relationship, and speaker.

[0021] Optionally, S2 is as follows:

[0022] The audio and video linkage channel status is used as input to receive conversation events. The conversation events are aligned according to a unified time base, and the start and end of speech, overlapping speech, pauses, lip movements and eye movements of the conversation events are extracted to form a set of conversation events.

[0023] The session event set is judged, and the speaking confidence threshold, speaking energy value threshold, lip opening threshold and gaze pointing threshold are set. The participants who meet the thresholds are added to the initial candidate set, and the event timestamps of the participants are recorded.

[0024] The initial candidate set is compared with the main speaker, and the initial candidate set is filtered using position stability threshold and speaking duration threshold. The candidate scores of the participants are calculated, and the participants who do not meet the position stability threshold and speaking duration threshold are eliminated to form the candidate main speaker.

[0025] The candidates are ranked according to their scores to determine the candidates who will speak. The status of the candidates is generated, which includes their identity, location, orientation, speaking duration, speaking confidence, lip opening status, and eye direction.

[0026] The session events are categorized with the status of the candidate speakers, and event tags are generated. The event tags include short interruption tags, takeover tags, and Q&A tags. The event tags are then associated with the corresponding participants and event timestamps.

[0027] Output the status and event flags of the candidate speaker object.

[0028] Optionally, S3 specifically refers to:

[0029] Using the candidate speaker's state and event markers as input, a conversation transition intention prediction model is constructed. A unified time base is established and the candidate speaker's state and event markers are time-aligned to obtain an input sequence for inference. The conversation transition intention prediction model consists of a rhythmic preamble unit, a dynamic determination unit, and a displacement memory unit.

[0030] The input sequence is analyzed using a rhythm preamble unit to extract the start and end of speech, pause duration, sentence end delay and energy rhythm. A confidence threshold for pause slots is set, and the time intervals with confidence scores not lower than the confidence threshold for pause slots are marked as pause slots, and the pause slots are output.

[0031] The dynamic judgment unit is used to jointly judge the gaze pointing state and lip opening state of the candidate speaker. The speaker posture threshold is set, and the speaker posture score is calculated within the pause time slot. Candidate speakers whose speaker posture score is not lower than the speaker posture threshold are marked as having a valid speaker posture, and the speaker posture result is output.

[0032] The position and orientation of the candidate presenter are accumulated in chronological order using displacement memory units to form a displacement sequence. A displacement stability threshold and an entry threshold are set to determine the continuity and orientation stability of the displacement sequence before and after the pause time slot, and the entry trend is obtained.

[0033] The pause time slot, the speaker's posture result, and the entry trend are combined with the event marker to generate a control signal, an executable window, and a pre-configuration amplitude. When the event marker is "takeover" and the speaker's posture is established and a pause time slot exists, the control signal is set to "switch", the executable window is taken as the intersection of the pause time slot and the stable interval of the entry trend, and the pre-configuration amplitude is determined based on the amplitude of the displacement sequence and the spatial difference between the candidate speaker and the speaker. When the event marker is "short interruption" and the speaker's posture is not established, the control signal is set to "hold", the executable window is taken as the earliest pause time slot, and the pre-configuration amplitude is set to "small pre-position".

[0034] The timing result is composed of the control signal, the executable window, and the pre-matched amplitude, and then output as the timing result.

[0035] Optionally, S4 specifically refers to:

[0036] Using the timing results as input, the control signals, executable windows, and pre-configured amplitudes are analyzed. An orchestration context is established based on the linkage orchestration framework controlled by the main speaker object. The main speaker object is used as the control object for the action arrangement of the camera, microphone, and display. The linkage orchestration framework is a control framework that takes the timing results and the state of the main speaker object as input, and arranges the camera pre-positioning trajectory, microphone secondary acquisition channel, display layout, and main acquisition takeover sequence in an integrated manner under a unified time base and response delay constraints, and outputs a linkage transition plan. It consists of a camera pre-positioning planning unit, a microphone secondary acquisition configuration unit, a display layout arrangement unit, and a constraint verification unit.

[0037] The camera pre-positioning planning unit forms a camera pre-positioning trajectory based on the pre-positioning amplitude, restricts the start and end times of the camera pre-positioning trajectory within the executable window, determines the target pose and transition path of the camera pre-positioning trajectory, and enables the camera to complete the pre-positioning from the current camera position to the target camera position within the executable window;

[0038] The microphone secondary acquisition configuration unit associates the microphone secondary acquisition channel with the main speaker, sets the entry and exit times of the microphone secondary acquisition channel in the executable window, and specifies the direction and channel switching strategy of the microphone secondary acquisition channel to ensure the audibility of the candidate main speaker before the main acquisition takes over.

[0039] The display layout scheduling unit schedules the display layout and the main acquisition takeover order within the executable window. The main acquisition takeover order is set to complete the camera pre-positioning trajectory first, then execute the main acquisition takeover, and then complete the display layout switching. The entry and exit times of the display layout are limited to the executable window.

[0040] The constraint verification unit unifies all actions of the camera pre-positioning trajectory, microphone secondary acquisition channel and display layout under the constraints of the executable window. When the control signal is hold, it only generates a small pre-configuration of the camera pre-positioning trajectory and the standby arrangement of the microphone secondary acquisition channel, without triggering the main acquisition takeover and display layout switching. When the control signal is switch, it generates a complete arrangement including the camera pre-positioning trajectory, microphone secondary acquisition channel and display layout.

[0041] The camera pre-positioning trajectory, microphone secondary acquisition channel, display layout, and main acquisition takeover sequence are arranged in time within the executable window to form a linkage transition plan, which is then output.

[0042] Optional, S5 specifically includes:

[0043] By taking the linkage transition plan and executable window as input, and combining a unified time base and response delay, the entry and exit times of the camera pre-positioning trajectory, microphone secondary acquisition channel and display layout are analyzed to form an aligned computational context.

[0044] Within the executable window, candidate timings are constructed for the end time of the camera pre-positioning trajectory, the start time of the main acquisition takeover sequence, and the start time of the display layout switching. A rhythm alignment threshold is set, and the times that meet the rhythm alignment threshold are added to the rhythm alignment point candidate set.

[0045] The candidate set of rhythm alignment points is filtered, the cross-device positioning error is calculated, and based on the response delay and action execution boundary, candidates that allow the camera to complete the pre-positioning trajectory, the microphone secondary acquisition channel to enter a stable state, and the main acquisition takeover to be executable are retained within the executable window to determine the rhythm alignment point.

[0046] Expand the rhythm alignment point into a switching execution list, fill the camera pre-positioning trajectory completion time, main acquisition takeover trigger time and display layout switching trigger time with a unified time base, and add offsets to the trigger times of the camera, microphone and display for latency compensation.

[0047] When the executable window is insufficient to accommodate the complete arrangement of the linkage transition plan, a hold strategy is adopted to generate a switch execution list containing only minor pre-configuration and standby arrangements. When the executable window meets the complete arrangement of the linkage transition plan, a switch execution list containing camera pre-positioning trajectory, microphone secondary acquisition channel and display layout is generated and output.

[0048] Optional, S6 specifically includes:

[0049] Using control signals and switching execution lists as input, a target execution list decision context is established based on a unified time base. The control signals are then type-determined to identify hold and switch.

[0050] When the control signal is held, an empty target execution list is generated. The target execution list does not include the trigger time of the camera preset trajectory, the microphone secondary acquisition channel and the display layout, nor does it include the action of the main acquisition takeover. This is used to keep the state of the main subject and the state of the audio and video linkage channel unchanged.

[0051] When the control signal is switching, the switching execution list is set as the target execution list. The trigger times of the camera, microphone and display in the switching execution list are retained according to a unified time base, and the offset settings used for time delay compensation are retained.

[0052] Perform constraint validation on the target execution list. If the trigger time of the target execution list is not within the executable window, set the target execution list to an empty target execution list. If the trigger times of the target execution list are all within the executable window, maintain the current setting.

[0053] Perform consistency checks on the target execution list. Based on the speaker's status, check the camera, microphone, and displayed object pointers. If the object pointers are inconsistent with the speaker's status, set the target execution list to empty. If they are consistent, output the target execution list.

[0054] The beneficial effects of this invention are:

[0055] This proposal presents an improved method for predicting conversational transition intentions and generating executable windows. It employs a three-unit collaborative approach: rhythmic preamble, momentum determination, and displacement memory. Under a unified time base, the rhythmic preamble identifies pause slots based on speech start and end times, pause durations, and energy rhythms. Within these pause slots, the speaker's posture is jointly calculated using gaze direction and lip-sync. Simultaneously, a displacement sequence is generated based on position and orientation, and entry trends and stable intervals are determined. These results are fused with event markers to output hold / switch control signals, executable windows, and pre-configured amplitudes. Compared to existing recognition schemes based solely on speech thresholds, this improvement completes timing-based decisions within natural sentence endings and spatial stabilization periods, reducing false triggers and cross-frame jitter, and providing executable temporal and spatial boundaries for subsequent cross-device collaboration.

[0056] This proposal presents a novel method for coordinated orchestration and rhythm alignment. Using the speaker as the controlled object, it integrates camera pre-positioning trajectory, microphone secondary acquisition channels, display layout, and main acquisition takeover sequence within an executable window, uniformly constrained by response latency and action boundaries. Constraint verification ensures all actions are completed within the window. Then, based on device trigger-effectiveness latency, the rhythm alignment point is calculated, precisely aligning the trigger times of camera pre-positioning completion, main acquisition takeover, and display switching, and performing offset compensation to form a switching execution list. When the window is insufficient, it automatically degrades to hold, generating only minor pre-configurations and standby arrangements. This method differs from distributed sequential triggering schemes, emphasizing cross-device time consistency and audible transition periods, reducing image shake and audio-visual asynchrony.

[0057] This proposal puts forward a holistic audio-visual linkage control method that integrates acquisition, judgment, arrangement, and execution. First, a unified time base, control path, and response latency model are established to output the status of the audio-visual linkage channels. Under a hold / switch strategy, the switching execution list is converted into a target execution list after consistency and window verification. Based on this, node control commands for camera PTZ, microphone main acquisition, and display layout are issued, and the status of the presenting object is written back. The overall design defines the executable window based on the intersection of pause / connection time slots, presenting posture, and entry trend, and implements cross-device collaboration using rhythm alignment points. This differs from existing methods that rely solely on threshold judgment and simple delay compensation, enabling the presenting transition in multi-person conversation scenarios to be completed within a controlled window, balancing transition smoothness and auditory continuity, and meeting the operational needs of actual broadcasting and conference acquisition. Attached Figure Description

[0058] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0059] Figure 1 This is a flowchart of a conference room audio and video control method based on artificial intelligence and audio-visual linkage arrangement proposed in this invention;

[0060] Figure 2 This invention presents a flowchart for aligning session events and screening candidate speakers in a conference room audio and video control method based on artificial intelligence and audio-visual linkage.

[0061] Figure 3 This is a flowchart illustrating the construction of a conversation transition intent prediction model for a conference room audio and video control method based on artificial intelligence and audio-visual linkage, as proposed in this invention.

[0062] Figure 4This invention presents a flowchart illustrating the construction of a collaborative arrangement framework for the speaker control in a conference room audio and video management method based on artificial intelligence and audio-visual linkage.

[0063] Figure 5 This is a schematic diagram illustrating the cross-device linkage of the executable window and rhythm alignment of a conference room audio and video control method based on artificial intelligence and audio-visual linkage arrangement proposed in this invention.

[0064] Figure 6 This is a top-down spatial linkage diagram of a conference room, illustrating a conference room audio-visual control method based on artificial intelligence and audio-visual linkage arrangement proposed in this invention. Detailed Implementation

[0065] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0066] refer to Figures 1 to 6 A method for controlling audio and video in a conference room based on artificial intelligence and audio-visual linkage, comprising:

[0067] S1. Obtain audio and video nodes and participants, establish an audio and video linkage channel and set the main speaker, and output the status of the audio and video linkage channel.

[0068] S2. Receive session events based on the status of the audio-visual linkage channel, determine the candidate speaker, and output the status and event flags of the candidate speaker.

[0069] S3. Based on the candidate speaker's status and event markers, construct a conversation transition intention prediction model. The conversation transition intention prediction model includes a rhythm preamble unit, a momentum determination unit, and a displacement memory unit. The rhythm preamble unit determines the pause time slots, the momentum determination unit combines line of sight and lip movement to identify the speaker's posture, the displacement memory unit estimates the entry trend, generates control signals, executable windows, and pre-configured amplitudes, and outputs timing results.

[0070] S4. Based on the timing results, a linkage transition plan is generated using a linkage arrangement framework controlled by the main speaker object: the camera pre-position trajectory is formed according to the pre-allocation amplitude, the microphone secondary acquisition channel is configured, the display layout and the main acquisition takeover order are arranged within the executable window, and the actions of the camera, microphone and display are constrained by the executable window, and the linkage transition plan is output.

[0071] S5. Calculate the rhythm alignment point and output the switching execution list based on the linkage transition plan and executable window;

[0072] S6. Determine the target execution list based on control signals;

[0073] S7. Generate node control instructions based on the target execution list to drive the linkage of camera PTZ, microphone main acquisition and display layout, and output the status of the main subject.

[0074] In this embodiment, step S1 specifically includes:

[0075] Using the camera's PTZ sensor, microphone, and display as audio / video nodes, capabilities and control constraints are acquired one by one. Preset position information and achievable angle range are collected for the camera's PTZ sensor, forming a set that includes the pan / tilt head's horizontal angle, pitch angle, and zoom range. Simultaneously, it records the motor's start-stop characteristics and speed limits to determine the boundaries of subsequent actions, and collects microphone pointing and switchable channels to form a set containing beam number, pointing azimuth angle, and channel identifier. The display capture can utilize various layouts and switching methods to create a collection that includes single-person scenes, double-person scenes, and picture-in-picture. The corresponding switching methods are then used to summarize the audio and video node information. This is used to characterize the control capabilities and limitations of each audio and video node;

[0076] The location and orientation of the participants were collected, and their positions were represented using a room coordinate system. The orientation of the participants is represented as The standing areas for participants are divided based on the effective activity area of ​​the room. Identify the movement status of participants based on position changes at adjacent times. Within a fixed time window, the displacement and orientation changes are statistically analyzed for subsequent stability assessment, based on audio and video node information. Establish a spatial mapping relationship with the location and orientation of the participants. , The positions and orientations of the participants are mapped to candidate preset positions for the camera's PTZ and candidate pointing positions for the microphone, and this is used to display the positions from... Select the display layout corresponding to the participants and generate participant information. This is used to describe the spatial correspondence between participants and audio / video nodes;

[0077] With audio and video node information Information on participants Then, an audio-visual linkage channel was established using both methods. Configure a unified time base With control path , Used for unified calibration of all trigger times and state samples. Used to transmit control commands from the control terminal to the camera PTZ, microphone, and display, measure the response delay of the audio-visual linkage channel, and record the response delay of the camera PTZ. Microphone response delay Response latency of the display The response delays are used as the basis for compensation calculations at subsequent trigger times. Based on motor start-stop characteristics, channel switching jitter time, and display switching duration, the action execution boundaries are determined. , This is used to constrain the minimum start-up advance and maximum allowable timeout of the camera's PTZ, to constrain the minimum stable duration of microphone channel switching, and to constrain the minimum duration of display switching, thereby forming an audio-visual linkage channel with a unified time base, control path, response delay, and action execution boundaries. ;

[0078] In the audio and video linkage channel After the setup is complete, set the speaking duration threshold. With position stability threshold Based on the information of the participants Using the channel energy statistics from the audio side as input, the speaking duration for each participant is calculated. With position stability Using spatial mapping relationships The participants were correlated with the microphone channel energy, and statistics were compiled over a unified time period. The duration of consecutive valid speech segments exceeding the energy threshold is used as... The variance of location and the magnitude of change in orientation within the statistical time window are used as... ,when Not less than and Not less than At that time, the corresponding participants will be set as the main speaker. If there are multiple participants who meet the criteria, then according to... The maximum value is used to uniquely determine the subject of the presentation. This is used to initialize the subsequent linkage control of the main speaker object;

[0079] Finally, the audio and video linkage channel Audio and video node information Information on participants With the main audience Summarize and output the status of the audio and video linkage channels. The status of the audio / video linkage channel Includes a unified time base Control path Response delay set Spatial mapping relationship With the main audience , It is used to support the subsequent reception of session events, the determination of candidate speakers and the invocation of the session transition intention prediction model, and to provide constraint benchmarks and object pointing benchmarks for the linkage and arrangement of camera PTZ, microphone and display on the same time axis;

[0080] Through the above steps, the acquisition of audio and video node information and participant information, the construction of audio and video linkage channels, and the setting of the speaker are completed, thus ensuring the status of the audio and video linkage channels. It possesses complete temporal, spatial, and control elements.

[0081] In this embodiment, step S2 specifically includes:

[0082] Status of audio and video linkage channels As input, the unified time base is analyzed. spatial mapping relationship ,exist The system receives conversation event streams from both the audio and video sides, performs time alignment on the audio side for the start and end of speech and overlapping speech, and performs time alignment on the video side for lip movements and gaze shifts, thus forming a set of conversation events. Each session event Includes start time and the end time and in Internally linked to attendee information Used for object indexing;

[0083] For the set of session events Make a judgment and set a speaking confidence threshold. Speaking energy threshold lip opening threshold With line of sight threshold , with session events As input, calculate the participants. exist Confidence of speech within Speaking energy Lip opening strength With line of sight intensity ,when and And exist make and At that time, the participants Add to the initial candidate set And record the attendees. Corresponding event timestamp Used for subsequent sorting and classification;

[0084] The initial candidate set With the main audience A comparison was performed using a position stability threshold. With the speaking duration threshold The initial candidate set is filtered, and the participants are calculated. Duration of the speech With position stability ,when and If it is valid, retain it; otherwise, discard it.

[0085] Calculate candidate scores for the retained participants. A weighted decision model is adopted:

[0086] ;

[0087] in In the audio and video linkage channel Fixed weights assigned by subscripts and This is the average value of the conversational event intervals, used to comprehensively assess the dominance of spoken and nonverbal cues;

[0088] Based on candidate scores Sort and select The largest value is selected as the candidate speaker, and the candidate speaker state is generated. , Includes identity ,Location Orientation Duration of the speech Confidence level of speech Lip opening state With eye pointing state ,in when make ,otherwise , when make ,otherwise ,Will With session events Summarize the results for event type determination;

[0089] The session events are categorized according to the status of the candidate speaker, event tags are generated and associated with the participants and the event timestamp, and the judgment rules for short interruption tags are defined as follows: and and Set the event flag to a short interrupt flag and associate it. The rule for determining the takeover flag is defined as follows: and and and in There are nearby objects from the main speaker. To the participants The focus shifts, the event flag is set as the takeover flag and associated. The rule for determining the question-and-answer tag is that within a fixed time window, there exists a candidate speaker and a speaker. The alternation of speeches between them, that is and And the start and end of their speeches were The following are arranged alternately, at the same time There is a bidirectional relationship between the two, and the event tag is set as a question-and-answer tag and associated. ;

[0090] Under a unified time base The above judgment results are then checked for time consistency to eliminate the offset caused by cross-modal sampling difference. Events that meet the time consistency criteria are then marked and the candidate presenter object status is output.

[0091] Through the above process, a linear processing chain is completed, from receiving and aligning session events, constructing the initial candidate set, comparing and screening with the main speaker, sorting candidate scores, to classifying event tags, resulting in the output candidate main speaker status. With event markers in The status of the audio and video linkage channel below Maintain consistency to provide complete, executable input for invoking the session transition intent prediction model.

[0092] In this embodiment, step S3 specifically includes:

[0093] Status of candidate speakers With event flags as input, in a unified time base The following establishes a time alignment process, Identity and position in Orientation Duration of the speech Confidence level of speech Lip opening state With eye pointing state Samples are taken in chronological order to form the input sequence. Mark the event in Expand below to range labels, and with Time alignment is performed to obtain the set of input sequences used for inference. ,by The event is tagged as input to the session transition intent prediction model;

[0094] The conversation transition intent prediction model consists of a rhythm preamble unit, a momentum determination unit, and a displacement memory unit. The rhythm preamble unit uses the speaking confidence level in the input sequence. With energy envelope As direct input, it includes an input buffer and rhythm feature extraction process. It extracts the start and end times of speech, pause duration, and sentence end delay according to a fixed duration window, and calculates the confidence level of the pause interval as follows:

[0095] ;

[0096] in The interval between adjacent speech segments, For the delay at the end of the sentence, Energy rhythm intensity, To achieve a unified time base The weight assigned by the index;

[0097] Disconnection slot confidence threshold Data was acquired through pre-deployment calibration and statistical analysis of samples from a fixed duration of no-handover data. The distribution of quantiles As ,when The time stamp is defined as the stop time slot, and the set of stop time slots is output. ;

[0098] The dynamic determination unit uses line-of-sight pointing state. With lip-syncing As a direct input, it includes a line-of-sight trajectory analysis and lip-shape intensity estimation process, during pauses. Internal calculation of the presenter's posture score:

[0099] ;

[0100] in and The average time within the pause slot. As a measure of alignment with the target audience's reference direction, In order to be in The weights assigned by the index, and the pose threshold are discussed in detail. The threshold search method using labeled samples is employed to select the appropriate threshold under a given false alarm rate constraint. Maximize the threshold as ,when Once the presenter's posture is established, output the presenter's posture result. ;

[0101] Displacement memory unit uses position Orientation As a direct input, it contains a time index and displacement accumulation process, which accumulates displacement sequences in chronological order. With orientation change sequence Calculate stability metrics before and after the disconnection time slot. and Displacement stability threshold The calibration data obtained using device space mapping will be the position variance quantiles in static sampling. As ,when and The time scale is defined as the stable interval, and the threshold is entered. The minimum perceptible displacement distance is obtained in the room coordinate system, calculated based on the camera's PTZ angular resolution and the microphone's pointing resolution. ,when Furthermore, if the orientation change is within a stable range, an entry trend is determined, and the entry trend is output. ;

[0102] Set up the pause time slots Main speaker's posture and results With the trend of entering By combining event markers, control signals, executable windows, and pre-configured amplitudes are generated. The determination is based on rule mapping; when an event is marked as takeover and Established and Non-empty and When established, switch Otherwise, set to maintain Executable window The calculation is as follows ,in This is the set of stable intervals output by the displacement memory unit;

[0103] Pre-configuration range The calculation is as follows:

[0104] ;

[0105] in The average displacement within the pause time slot. The angular difference between the camera's PTZ and the target camera position, mapped from the current camera position to the candidate subject space. For microphone pointing adjustment, The weights are assigned at the action execution boundary when the event is marked as a short interruption and If not, maintain , Take the earliest disconnection time slot. Set to small pre-position, , and Composition timing results Output Used for subsequent linkage and switching arrangements;

[0106] Through the above process, linear inference is achieved from input alignment, rhythm preamble, dynamic determination to displacement memory and fusion control, ensuring that the control signal, executable window, and pre-matched amplitude are on a unified time base. This can be used directly for subsequent execution.

[0107] In this embodiment, step S4 specifically includes:

[0108] Timing results Status of the presenter As input, combined with the audio / video linkage channel status Unified Time Base Response delay set With action execution boundary Analyze control signals, executable windows, and pre-configured amplitudes to establish a configuration context. The executable window will be displayed below. The standard is When maintaining At that time, only a small amount of pre-provisioning and standby arrangements are generated, and when switching... At that time, the complete allocation process begins;

[0109] The camera prepositioning planning unit is based on the prepositioning amplitude. Forming a camera pre-positioning trajectory based on the state of the subject being presented. spatial mapping relationship Calculate target pose The current pose arrive The transition path is represented as The use of speed limits and angular acceleration limits originates from ,exist Set start time and the end time ,in in accordance with and The maximum permissible speed is calculated to ensure exist Completed internally by arrive Pre-positioning, and ensuring that the angle change does not exceed Limited range;

[0110] The microphone secondary acquisition configuration unit connects the microphone secondary acquisition channel with the status of the main speaker. To link, based on Determine the pointing vector ,exist Set entry time With exit time ,in in accordance with The shortest stable duration is determined, and the channel switching strategy adopts the approach of first pointing the secondary acquisition channel to... ,exist Activate the secondary acquisition channel and maintain it until... This is used to ensure the audibility of candidate speakers before the main acquisition takes over; all pointing and switching operations are performed in accordance with... and Perform time delay compensation;

[0111] Display layout arrangement units in the executable window The internal layout and main acquisition takeover sequence are defined. The main acquisition takeover sequence is set to complete the camera pre-positioning trajectory first, then execute the main acquisition takeover, and finally complete the display layout switching. The trigger time for the main acquisition takeover is defined. Display layout switching trigger time ,in In order to be in The specified takeover protection interval, used to prevent concurrent jitter between the camera and display, will determine the entry time of the display layout. With exit time ,in Determined based on the shortest duration of the display switching;

[0112] The constraint verification unit performs consistency and window constraint verification on all actions of the camera pre-positioning trajectory, microphone secondary acquisition channel and display layout;

[0113] Using window constraint checking function when ,otherwise ,right Check each one individually; if it is not satisfied at any given time... At that time, if maintained Only minor camera setup and microphone standby arrangements are retained, while the main acquisition takeover and display layout switching are removed. (If switching...) Then, the out-of-bounds moments are pruned and adjacent intervals are recalculated to ensure all moments fall within the bounds. And maintain the order relationship ;

[0114] After the timing sequence and parameters generated by the above units pass the constraint verification, the camera pre-positioning trajectory will be... Microphone secondary acquisition channel entry and exit timing Display layout switching sequence Triggering time of main data acquisition takeover exist The following timelines will be arranged to form a coordinated transition plan. When maintaining hour, Only contains The minor pre-configuration and standby timing of the secondary acquisition channel do not include... With display layout switching, when switching hour, Includes complete camera pre-positioning trajectory, microphone secondary acquisition channel timing, main acquisition takeover triggering and display layout switching, and finally outputs a linkage transition plan. Used for unified time base With response latency The subsequent rhythm alignment and switching execution are driven by constraints.

[0115] In this embodiment, step S5 specifically includes:

[0116] Linked transition plan With executable window As input, combined with a unified time base With response delay set Analyze the end time of the camera pre-positioning trajectory Entry and exit times of the microphone secondary acquisition channel and the start time of the display layout switch Simultaneously analyze the start time of the main acquisition takeover sequence. This forms an alignment computation context for subsequent candidate construction and filtering;

[0117] Candidate construction is performed within the executable window, taking the alignment computation context as input, and defining the alignment candidate set. It consists of three types of moments: the end moment of the camera pre-positioning trajectory. The start time of the main acquisition takeover sequence The start time of display layout switching For each candidate time The rhythm alignment score is calculated using a scoring function:

[0118] ;

[0119] in:

[0120] ;

[0121] ;

[0122] ;

[0123] To be at the action execution boundary The minimum stable duration of the microphone specified by the subscript. For timescale parameters calibrated under a unified time base, the rhythm alignment threshold is... Obtained through the alignment calibration set before deployment, Quantiles on the calibration set Set as ,when and When that moment is reached, add it to the rhythm alignment point candidate set. ;

[0124] Candidate set of rhythm alignment points Perform screening and calculate cross-device positioning error. ,when And satisfy the action execution boundary Candidates are retained during constraint, where The quantiles of the error distribution were obtained by statistically analyzing the error distribution of historical switchover samples. Set as Simultaneously check the stable range of the microphone secondary acquisition channel. Does it cover? And check the camera pre-positioning trajectory in The previous steps have been completed, and the display layout switch is now in progress. The timing is determined by ensuring that the triggering conditions are met at all times, and the moment with the smallest error and all constraints are satisfied is taken as the rhythm alignment point. ;

[0125] Align the rhythm point Expanding to a switching execution list, offsets are added to the trigger times of the camera, microphone, and display for latency compensation. Let the camera pre-positioning trajectory completion time be... The camera triggers the download at the following time. Let the trigger time for the main data acquisition takeover be... The microphone triggers the sending time. When the main acquisition receiver requires a protection interval, it is used in The specified protection interval To perform calibration, let the display layout switching trigger time be... ,Will Based on a unified time base Fill in the list to create a switch execution list. ;

[0126] When the executable window is insufficient to accommodate the complete arrangements of the linkage transition plan, a hold strategy is adopted to generate a switchover execution list containing only a small amount of pre-provisioning and standby arrangements;

[0127] For the required duration As a basis for judgment, when At that time, set Includes only minor camera pre-configuration trigger times Microphone secondary acquisition ready to enter time No main acquisition takeover trigger or display layout switching trigger is generated. When the executable window meets the complete arrangement of the linkage transition plan, a switching execution list containing the camera pre-positioning trajectory completion time, the main acquisition takeover trigger time, and the display layout switching trigger time is generated. The output is used for issuing subsequent node control commands.

[0128] In this embodiment, step S6 specifically includes:

[0129] control signal With switching execution list As input, combined with the audio / video linkage channel status Unified Time Base With executable window Establish a decision context for the target execution checklist, for Analyze the data and extract the camera trigger time. Master data acquisition takeover trigger time Display layout switching trigger time and offset configuration for latency compensation. ,exist The above times are uniformly calibrated to form a sequence to be determined. ;

[0130] The control signal is type-determined, and a mapping rule is used to maintain the switching. When maintaining At that time, an empty target execution list is generated. , It does not include the trigger times for camera pre-positioning trajectory, microphone secondary acquisition channel, and display layout, nor does it include the main acquisition takeover action. It is used to maintain the status of the main speaker and the audio / video linkage channel unchanged when switching... At that time, Set as the initial target execution list ,reserve and The settings ensure that latency compensation remains consistent in subsequent distribution stages;

[0131] Constraint validation is performed on the target execution list using a window decision function. when ,otherwise When there is make At that time, directly Set as When all trigger times are satisfied At that time, maintain The current setting also incorporates jitter tolerance. Tolerance for time sampling errors. The minimum start-up advance in the action execution boundary is determined by the sum of the sampling period. ,in Calibrated by the device drive specifications, for The sampling period is as follows, when This is considered to satisfy the window boundary;

[0132] Perform consistency checks on the target execution list based on the status of the main object. Verify that the camera, microphone, and display are pointing at the correct objects;

[0133] Using spatial mapping relationship Calculate the desired object pointing direction to obtain the camera target pose. Microphone target direction With display target layout ,exist Read the target pose of the camera's current pre-positioning trajectory from the device configuration context. Target direction of microphone secondary acquisition channel Target configuration with display layout The object being calculated points to the error vector. , Inconsistency with layout indicators Set the threshold for the object to point to. To determine consistency, The data is obtained based on the camera's PTZ angular resolution and the microphone's pointing resolution calibration. ,when and and At that time, the state of the object being judged is consistent with that of the object being presented, and this is maintained. Unchanged; when any condition is not met, it will... Set as ;

[0134] To ensure that the latency compensation settings are fully preserved, for The trigger time is used to apply offset consistency check and calculate , ,when If the compensation configuration is consistent, then... Set as The above Used to verify the triggering and effective times. The alignment relationship below, The source has been disclosed in the window verification;

[0135] exist After passing window constraints and consistency checks, the target execution list is output. ,when At this time, no subsequent node control commands will be issued, and the state of the main speaker and the audio / video linkage channel will remain unchanged. hour, Include and its corresponding In a unified time base The following is used to drive the linkage between camera PTZ, microphone main acquisition, and display layout;

[0136] Through the linear process of decision context construction, type determination, window constraint verification, and object pointer consistency check, the switching execution list is retained as the target execution list only when both time and pointer constraints are met, thereby providing strict timing and object consistency guarantees for the stable issuance of subsequent node control instructions.

[0137] In this embodiment, step S7 specifically includes:

[0138] Target execution list Status of audio and video linkage channels As input, in a unified time base Analysis below Camera triggering the release time Master data acquisition takeover trigger time Triggering time of display layout switching Read the response latency set Control path spatial mapping relationship And read the state of the presenter object. ,when At that time, no node control instructions that change the state are generated, and the state is maintained. Unchanged and directly output, when At that time, the instruction generation process begins;

[0139] Generate node control commands for camera PTZ, and set the state of the main object. spatial mapping relationship As input, calculate the target pose. Read the current pose And calculate the angle difference Set the camera speed limit The upper limit of angular acceleration is derived from the action execution boundary, and the expected arrival time is calculated. , Take as the smallest controllable time granularity The sampling period is used to construct the camera PTZ control command load. ,exist Always along the control path Distribute and follow Alignment is performed using the reference time for arrival.

[0140] Control commands for the microphone master acquisition and generation node, based on Calculate the target direction of the main acquisition Determine the main acquisition target channel index Set the fade-in and fade-out duration for channel handover. Based on the shortest stable duration of the action execution boundary, the time of the main data acquisition takeover is set to... ;

[0141] Construct the microphone master acquisition control command load ,exist Distribute, and use a weight function Used for energy weight interpolation within the takeover window to ensure that... The subsequent ruler took complete control of the data collection process.

[0142] Control commands for generating nodes in the display layout are generated in accordance with... and Determine the target display layout The display switching style and minimum duration are set based on the action execution boundary. Time Construction and along the control path Issued, with As a reference point for when the layout takes effect;

[0143] The system checks the trigger alignment and jitter, and sets an alignment tolerance. The effective deviation is calculated as the sum of the minimum start-up advance and the sampling period. and ,when At that time, a one-time fine adjustment is performed, shifting the release time of the camera or display respectively. The command was then issued again to ensure that the camera was in place, the main acquisition takeover was in place, and the display switching was in place. Land within the same aligned window;

[0144] The node control commands are summarized and sorted. , and according to Enqueue in ascending order, using a control path Send out each item individually and record the arrival time of the device receipt. Used for subsequent state update determination, let the success condition be: , and ;

[0145] Output the status of the main object, if the target execution list is... Then output the original presenter's state. ,like And if all of the above success conditions are met simultaneously, then in Update the speaker's status to ,in The identity of the participant is obtained by inferring the target direction from the microphone, using information from the participant information set that meets the criteria. The only attendee As a new identity The position and orientation are set to the maximum values ​​of the camera's PTZ angular resolution and the microphone's pointing resolution, and then taken from these values. The current situation It inherits the speaking confidence and speaking duration from the latest session event statistics and outputs the updated data. If any success condition is not met, then maintain It remains unchanged and triggers a small pre-allocated compensation distribution until it enters a new executable window before taking over.

[0146] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for audio and video control in a conference room based on artificial intelligence and audio-visual linkage, characterized in that, Includes the following steps: S1. Obtain audio and video nodes and participants, establish an audio and video linkage channel and set the main speaker, and output the status of the audio and video linkage channel. S2. Receive session events based on the status of the audio-visual linkage channel, determine the candidate speaker, and output the status and event flags of the candidate speaker. S3. Based on the candidate speaker's status and event markers, construct a conversation transition intention prediction model. The conversation transition intention prediction model includes a rhythm preamble unit, a momentum determination unit, and a displacement memory unit. The rhythm preamble unit determines the pause time slots, the momentum determination unit combines line of sight and lip movement to identify the speaker's posture, the displacement memory unit estimates the entry trend, generates control signals, executable windows, and pre-configured amplitudes, and outputs timing results. S4. Based on the timing results, a linkage transition plan is generated using a linkage arrangement framework controlled by the main speaker object: the camera pre-position trajectory is formed according to the pre-allocation amplitude, the microphone secondary acquisition channel is configured, the display layout and the main acquisition takeover order are arranged within the executable window, and the actions of the camera, microphone and display are constrained by the executable window, and the linkage transition plan is output. S5. Calculate the rhythm alignment point and output the switching execution list based on the linkage transition plan and executable window; S6. Determine the target execution list based on control signals; S7. Generate node control instructions based on the target execution list to drive the linkage of camera PTZ, microphone main acquisition and display layout, and output the status of the main subject.

2. The conference room audio and video control method based on artificial intelligence and audio-visual linkage arrangement according to claim 1, characterized in that, S1 specifically refers to: Acquire preset position information and reachable angle range of camera PTZ, acquisition direction and switchable channels of microphone, available display layout and switching mode, and generate audio and video node information; Collect the location and orientation of the participants, obtain the standing area and movement status of the participants, establish the spatial mapping relationship between the participants and audio and video nodes, and generate participant information; An audio-visual linkage channel is established by using audio and video node information and participant information, a unified time base and control path are configured, and the response latency and action execution boundary of the audio-visual linkage channel are measured to form an audio-visual linkage channel. Set a speaking duration threshold and a position stability threshold, and set the participants who meet the speaking duration threshold and position stability as the main speakers; The audio-visual linkage channel, audio-visual node information, participant information, and speaker information are summarized to output the audio-visual linkage channel status. The audio-visual linkage channel status includes a unified time base, control path, response delay, spatial mapping relationship, and speaker.

3. The conference room audio and video control method based on artificial intelligence and audio-visual linkage arrangement according to claim 1, characterized in that, S2 specifically refers to: The audio and video linkage channel status is used as input to receive conversation events. The conversation events are aligned according to a unified time base, and the start and end of speech, overlapping speech, pauses, lip movements and eye movements of the conversation events are extracted to form a set of conversation events. The session event set is judged, and the speaking confidence threshold, speaking energy value threshold, lip opening threshold and gaze pointing threshold are set. The participants who meet the thresholds are added to the initial candidate set, and the event timestamps of the participants are recorded. The initial candidate set is compared with the main speaker, and the initial candidate set is filtered using position stability threshold and speaking duration threshold. The candidate scores of the participants are calculated, and the participants who do not meet the position stability threshold and speaking duration threshold are eliminated to form the candidate main speaker. The candidates are ranked according to their scores to determine the candidates who will speak. The status of the candidates is generated, which includes their identity, location, orientation, speaking duration, speaking confidence, lip opening status, and eye direction. The session events are categorized with the status of the candidate speakers, and event tags are generated. The event tags include short interruption tags, takeover tags, and Q&A tags. The event tags are then associated with the corresponding participants and event timestamps. Output the status and event flags of the candidate speaker object.

4. The conference room audio and video control method based on artificial intelligence and audio-visual linkage arrangement according to claim 1, characterized in that, S3 specifically refers to: Using the candidate speaker's state and event markers as input, a conversation transition intention prediction model is constructed. A unified time base is established and the candidate speaker's state and event markers are time-aligned to obtain an input sequence for inference. The conversation transition intention prediction model consists of a rhythmic preamble unit, a dynamic determination unit, and a displacement memory unit. The input sequence is analyzed using a rhythm preamble unit to extract the start and end of speech, pause duration, sentence end delay and energy rhythm. A confidence threshold for pause slots is set, and the time intervals with confidence scores not lower than the confidence threshold for pause slots are marked as pause slots, and the pause slots are output. The dynamic judgment unit is used to jointly judge the gaze pointing state and lip opening state of the candidate speaker. The speaker posture threshold is set, and the speaker posture score is calculated within the pause time slot. Candidate speakers whose speaker posture score is not lower than the speaker posture threshold are marked as having a valid speaker posture, and the speaker posture result is output. The position and orientation of the candidate presenter are accumulated in chronological order using displacement memory units to form a displacement sequence. A displacement stability threshold and an entry threshold are set to determine the continuity and orientation stability of the displacement sequence before and after the pause time slot, and the entry trend is obtained. The pause time slot, the speaker's posture result, and the entry trend are combined with the event marker to generate a control signal, an executable window, and a pre-configuration amplitude. When the event marker is "takeover" and the speaker's posture is established and a pause time slot exists, the control signal is set to "switch", the executable window is taken as the intersection of the pause time slot and the stable interval of the entry trend, and the pre-configuration amplitude is determined based on the amplitude of the displacement sequence and the spatial difference between the candidate speaker and the speaker. When the event marker is "short interruption" and the speaker's posture is not established, the control signal is set to "hold", the executable window is taken as the earliest pause time slot, and the pre-configuration amplitude is set to "small pre-position". The timing result is composed of the control signal, the executable window, and the pre-matched amplitude, and then output as the timing result.

5. The conference room audio and video control method based on artificial intelligence and audio-visual linkage arrangement according to claim 1, characterized in that, S4 specifically refers to: Using the timing results as input, the control signals, executable windows, and pre-configured amplitudes are analyzed. An orchestration context is established based on the linkage orchestration framework controlled by the main speaker object. The main speaker object is used as the control object for the action arrangement of the camera, microphone, and display. The linkage orchestration framework is a control framework that takes the timing results and the state of the main speaker object as input, and arranges the camera pre-positioning trajectory, microphone secondary acquisition channel, display layout, and main acquisition takeover sequence in an integrated manner under a unified time base and response delay constraints, and outputs a linkage transition plan. It consists of a camera pre-positioning planning unit, a microphone secondary acquisition configuration unit, a display layout arrangement unit, and a constraint verification unit. The camera pre-positioning planning unit forms a camera pre-positioning trajectory based on the pre-positioning amplitude, restricts the start and end times of the camera pre-positioning trajectory within the executable window, determines the target pose and transition path of the camera pre-positioning trajectory, and enables the camera to complete the pre-positioning from the current camera position to the target camera position within the executable window; The microphone secondary acquisition configuration unit associates the microphone secondary acquisition channel with the main speaker, sets the entry and exit times of the microphone secondary acquisition channel in the executable window, and specifies the direction and channel switching strategy of the microphone secondary acquisition channel to ensure the audibility of the candidate main speaker before the main acquisition takes over. The display layout scheduling unit schedules the display layout and the main acquisition takeover order within the executable window. The main acquisition takeover order is set to complete the camera pre-positioning trajectory first, then execute the main acquisition takeover, and then complete the display layout switching. The entry and exit times of the display layout are limited to the executable window. The constraint verification unit unifies all actions of the camera pre-positioning trajectory, microphone secondary acquisition channel and display layout under the constraints of the executable window. When the control signal is hold, it only generates a small pre-configuration of the camera pre-positioning trajectory and the standby arrangement of the microphone secondary acquisition channel, without triggering the main acquisition takeover and display layout switching. When the control signal is switch, it generates a complete arrangement including the camera pre-positioning trajectory, microphone secondary acquisition channel and display layout. The camera pre-positioning trajectory, microphone secondary acquisition channel, display layout, and main acquisition takeover sequence are arranged in time within the executable window to form a linkage transition plan, which is then output.

6. The conference room audio and video control method based on artificial intelligence and audio-visual linkage arrangement according to claim 1, characterized in that, S5 specifically refers to: By taking the linkage transition plan and executable window as input, and combining a unified time base and response delay, the entry and exit times of the camera pre-positioning trajectory, microphone secondary acquisition channel and display layout are analyzed to form an aligned computational context. Within the executable window, candidate timings are constructed for the end time of the camera pre-positioning trajectory, the start time of the main acquisition takeover sequence, and the start time of the display layout switching. A rhythm alignment threshold is set, and the times that meet the rhythm alignment threshold are added to the rhythm alignment point candidate set. The candidate set of rhythm alignment points is filtered, the cross-device positioning error is calculated, and based on the response delay and action execution boundary, candidates that allow the camera to complete the pre-positioning trajectory, the microphone secondary acquisition channel to enter a stable state, and the main acquisition takeover to be executable are retained within the executable window to determine the rhythm alignment point. Expand the rhythm alignment point into a switching execution list, fill the camera pre-positioning trajectory completion time, main acquisition takeover trigger time and display layout switching trigger time with a unified time base, and add offsets to the trigger times of the camera, microphone and display for latency compensation. When the executable window is insufficient to accommodate the complete arrangement of the linkage transition plan, a hold strategy is adopted to generate a switch execution list containing only minor pre-configuration and standby arrangements. When the executable window meets the complete arrangement of the linkage transition plan, a switch execution list containing camera pre-positioning trajectory, microphone secondary acquisition channel and display layout is generated and output.

7. The conference room audio and video control method based on artificial intelligence and audio-visual linkage arrangement according to claim 1, characterized in that, S6 specifically refers to: Using control signals and switching execution lists as input, a target execution list decision context is established based on a unified time base. The control signals are then type-determined to identify hold and switch. When the control signal is held, an empty target execution list is generated. The target execution list does not include the trigger time of the camera preset trajectory, the microphone secondary acquisition channel and the display layout, nor does it include the action of the main acquisition takeover. This is used to keep the state of the main subject and the state of the audio and video linkage channel unchanged. When the control signal is switching, the switching execution list is set as the target execution list. The trigger times of the camera, microphone and display in the switching execution list are retained according to a unified time base, and the offset settings used for time delay compensation are retained. Perform constraint validation on the target execution list. If the trigger time of the target execution list is not within the executable window, set the target execution list to an empty target execution list. If the trigger times of the target execution list are all within the executable window, maintain the current setting. Perform consistency checks on the target execution list. Based on the speaker's status, check the camera, microphone, and displayed object pointers. If the object pointers are inconsistent with the speaker's status, set the target execution list to empty. If they are consistent, output the target execution list.

Images