Detection method and system for testing sound image co-location capability of immersive video conference

Abstract

The invention relates to a detection method and system for testing the sound image co-location capability of an immersive video conference, and belongs to the technical field of immersive video conferences. The method comprises the following steps: in a conference room, synchronously sending audio and video signals in a moving process by using a signal generating device, and recording sending position information and time parameters; in the other conference room, respectively recording corresponding receiving time parameters based on the received audio played by the audio player and the received video played by the video player, and carrying out sound source positioning to obtain sound source position information; and calculating an acoustic image co-location index value based on each transmit-receive parameter to obtain a detection result, and evaluating the acoustic image co-location capability of audio and video signal transmission between the two conference rooms. According to the method and the system, the immersive conference system can be efficiently and automatically tested, the evaluation result can be given, and the problem that an existing immersive conference system lacks an effective test means is solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of immersive video conferencing, and in particular relates to a detection method and system for testing the audio and video parity capability of immersive video conferencing. Background Art

[0002] An immersive video conferencing system is a video conferencing system that restores the conference scene to the greatest extent possible. Compared with traditional video conferencing systems, it provides users with a face-to-face video conferencing effect. In terms of video images, the immersive video conferencing system can restore the conference scene in proportion at the remote end; in terms of sound, the immersive video conferencing system can restore the stereoscopic effect of sound at the remote end. In addition to restoring the sound, it can also restore the location information of the sound. In the specific technical implementation process of the video conferencing system, video and audio are usually transmitted as independent media channels. In immersive video conferencing, due to the complexity of images and sounds, multiple channels are required to transmit video and audio respectively. Therefore, ensuring that images and sounds are played synchronously at the correct position (abbreviated as audio-visual synchronization) is a technical difficulty of the immersive video conferencing system and an important indicator for evaluating the quality of the immersive video conferencing system.

[0003] The audio and video colocation effect test of the immersive video conferencing system is usually conducted manually, with two testers in different venues receiving each other's audio and video signals to make manual judgments. One tester moves in the venue while sending out test sounds, while the other tester observes the movement trajectory of the remote tester and receives the sound in the venue, and judges the audio and video colocation effect by perception. This method is too subjective, and the test results are judged only by the testee's feelings, which cannot be quantified and has low accuracy. At the same time, manual testing methods cannot achieve test automation, and the test efficiency is low. Summary of the invention

[0004] In view of the above analysis, the present invention aims to provide a detection method and system for testing the audio and video parity capability of immersive video conferencing. Through the system automatically playing test sequences, receiving test sequences, listening to and analyzing the position of the audio, the audio and video parity effect of the immersive video conferencing is evaluated and a quantitative evaluation result is given, thereby avoiding the problems of low efficiency and inaccurate test results of traditional manual testing.

[0005] On the one hand, the present invention provides a detection method for testing the audio and video co-location capability of immersive video conferencing, which specifically includes the following steps:

[0006] In one of the conference rooms, a signal generating device is used to synchronously send out audio and video signals during the movement, and the corresponding sending position information and sending time parameters are recorded;

[0007] In another conference room, corresponding receiving time parameters are recorded based on the received audio played by the audio player and the received video played by the video player, and sound source positioning is performed based on the received audio to obtain sound source position information of the received audio;

[0008] Based on the sending position information, the sound source position information, the sending time parameter and the receiving time parameter, the sound and image co-location index value is calculated to obtain the detection result; wherein the sending position information and the sound source position information are the relative position information obtained with the corresponding positions as reference points in the two conference rooms respectively;

[0009] The audio and video co-location capability of the audio and video signal transmission between the two conference rooms is obtained based on the detection result.

[0010] Furthermore, the sound image parity index includes the mean value U of the sound position deviation value PL PL and variance S PL , the mean value U of the audio and video signal synchronization deviation from PSV PSV and variance S PSV , the mean value of the sound image deviation is U PLSV , respectively, using the following formulas to calculate:

[0011]

[0012]

[0013]

[0014]

[0015] U PLSV =ln(1+U PL )+ln(1+U PSV );

[0016] Among them, i represents the i-th signal emitted by the signal generating device, N represents the total number of signal occurrences; Li is the sound source location information, li is the sending location information; Si is the audio signal receiving time, and Vi is the image signal receiving time.

[0017] Further, the evaluation of the audio and video co-location capability of the audio and video signal transmission between the two conference rooms based on the detection result includes: when the sound signal deviation value is measured in centimeters and the audio and video synchronization time is measured in milliseconds:

[0018] 0 PLSV ≤6, the audio-visual synchronization ability is evaluated as high level, indicating that the audio-visual synchronization ability is good;

[0019] 6 PLSV ​​≤10, the audio-visual synchronization ability is evaluated as medium level, indicating that the audio-visual synchronization ability is acceptable;

[0020] Then U PLSV >10, the audio-visual synchronization ability is evaluated as low level, indicating that the audio-visual synchronization ability is poor.

[0021] Further, the use of the signal generating device to synchronously send out audio and video signals during the movement and record the corresponding first position information and sending time parameters includes:

[0022] A signal generating device is used to synchronously send out audio and video signals at intervals during the movement process, and the sending time parameters of each signal are recorded; an infrared signal ranging method is used to measure and obtain the sending position information of each signal sent by the signal generating device.

[0023] Furthermore, a microphone array is used to receive the received audio played by the audio player, and the sound source is located through the audio intensity and time difference received by each microphone to obtain the sound source position information of the received audio.

[0024] On the other hand, the present invention also provides a detection system for testing the audio and video co-location capability of immersive video conferencing, comprising: a signal generating end deployed in one of the conference rooms and a signal receiving and analyzing end deployed in another of the conference rooms;

[0025] The signal generating end includes a signal generating device, a position positioning module, a motion trajectory control module, and a first information synchronization module; wherein,

[0026] The signal generating device is used to synchronously send out audio and video signals during the movement and record the corresponding sending time parameters;

[0027] The motion trajectory control module is used to control the movement trajectory of the signal generating device;

[0028] The position positioning module is used to obtain the sending position information when the signal generating device sends out the audio and video signals;

[0029] The first information synchronization module is used to synchronize the sending position information, the audio and video signals and the corresponding sending time parameters to the signal receiving and analyzing end;

[0030] The signal receiving and analyzing end includes a signal detection module, a sound location identification module, a second information synchronization module and a result analysis module; wherein,

[0031] The signal detection module is used to detect corresponding reception time parameters based on the received audio played by the audio player in the conference room and the received video played by the video player;

[0032] The audio location module is used to locate the sound source based on the received audio played by the audio player in the conference room, and obtain the sound source position information of the received audio;

[0033] The second information synchronization module is used to receive information sent by the first information synchronization module;

[0034] The result analysis and presentation module is used to calculate the audio and video parity index value to obtain the detection result, evaluate the audio and video parity capability of the audio and video signal transmission between the two conference rooms based on the detection result, and present it to the user.

[0035] Furthermore, the sound and image parity index includes: the mean value U of the sound position deviation value PL PL and variance S PL , the mean value U of the audio and video signal synchronization deviation from PSV PSV and variance S PSV , the mean value of the sound image deviation is U PLSV , respectively, using the following formulas to calculate:

[0036]

[0037]

[0038]

[0039]

[0040] U PLSV =ln(1+U PL )+ln(1+U PSV );

[0041] Among them, i represents the i-th signal emitted by the signal generating device, N represents the total number of signal occurrences; Li is the sound source location information, li is the sending location information; Si is the audio signal receiving time, and Vi is the image signal receiving time.

[0042] Further, the evaluation of the audio and video co-location capability of the audio and video signal transmission between the two conference rooms based on the detection result includes: when the sound signal deviation value is measured in centimeters and the audio and video synchronization time is measured in milliseconds:

[0043] 0 PLSV ≤6, the audio-visual synchronization ability is evaluated as high level, indicating that the audio-visual synchronization ability is good;

[0044] 6 PLSV ≤10, the audio-visual synchronization ability is evaluated as medium level, indicating that the audio-visual synchronization ability is acceptable;

[0045] Then U PLSV ​​>10, the audio-visual synchronization ability is evaluated as low level, indicating that the audio-visual synchronization ability is poor.

[0046] Further, the signal generating device includes an audio signal generating module and a video signal generating module;

[0047] The signal generating device synchronously sends out audio and video signals during movement and records corresponding sending time parameters, including: during movement, the audio signal generating module and the video signal generating module synchronously send out multiple audio and video signals at intervals, and record the sending time parameters of each signal.

[0048] Furthermore, the audio positioning module uses a microphone array to receive the audio played by the audio player, and locates the sound source through the audio intensity and time difference received by each microphone to obtain the sound source position information of the received audio.

[0049] The present invention can achieve at least one of the following beneficial effects:

[0050] The audio and video signal generating end automatically sends out the test sequence signal, and the signal receiving and analyzing end receives the test sequence signal for analysis. Finally, the effect of audio and video colocation is represented by quantitative numerical indicators. This solves the problems of strong subjectivity and low efficiency caused by the traditional audio and video colocation test method that relies on manual judgment, and improves the test efficiency and the accuracy of the test results.

[0051] By calculating the mean value U of the sound position deviation value PL PL and variance S PL , the mean value U of the audio and video signal synchronization deviation from PSV PSV and variance S PSV , the mean value of the sound image deviation is U PLSV , use U PLSV Comprehensive evaluation of audio-visual parity, using U PL , S PL , U PSV , S PSV The calculated result value specifically locates the cause of the audio transmission or video transmission that may cause poor audio-visual co-location capabilities, so that the test results can assist in locating transmission problems in immersive video conferencing.

[0052] By using a regular test sequence of audio and video signals and analyzing the reception of multiple signals, the calculated evaluation index values ​​can scientifically and objectively reflect the audio and video colocation capabilities of immersive video conferencing, thereby improving the accuracy of detection and evaluation.

[0053] Other features and advantages of the present invention will be described in the following description, and some advantages may become apparent from the description, or may be understood by practicing the present invention. The purpose and other advantages of the present invention may be realized and obtained through the contents particularly pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The accompanying drawings are only used for the purpose of illustrating specific embodiments and are not to be considered as limiting the present invention. In the entire drawings, the same reference symbols represent the same components;

[0055] Figure 1 The present invention is a flow chart of a method for detecting the parity of sound and image;

[0056] Figure 2 A schematic diagram of the sending rules of audio and video in a method embodiment;

[0057] Figure 3 This is a schematic diagram of infrared signal ranging in a method embodiment;

[0058] Figure 4 A schematic diagram of detecting and analyzing the audio and video co-location capability of an immersive video conference in a method embodiment;

[0059] Figure 5 A schematic diagram of locating the sound source position of an immersive video conference in a method embodiment;

[0060] Figure 6 It is a schematic diagram of the structure of the detection system of the audio and video parity capability of the present invention;

[0061] Figure 7 The figure is a flow chart of the operation of the system for detecting the audio and video parity capability according to the system embodiment. DETAILED DESCRIPTION

[0062] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not used to limit the scope of the present invention.

[0063] Method Embodiment

[0064] A specific embodiment of the present invention discloses a detection method for testing the audio and video colocation capability of immersive video conferencing. The method of this embodiment is used to detect the audio and video colocation capability of two conference rooms built according to immersive video standard specifications. Specifically, it includes the following steps:

[0065] Step S01: In one of the conference rooms, a signal generating device is used to synchronously send out audio and video signals during movement, and corresponding sending position information and sending time parameters are recorded;

[0066] Step S02: in another conference room, corresponding receiving time parameters are recorded based on the received audio played by the audio player and the received video played by the video player, and sound source positioning is performed based on the received audio to obtain sound source position information of the received audio;

[0067] Step S03, based on the sending position information, the sound source position information, the sending time parameter and the receiving time parameter, the sound image parity index value is calculated to obtain a detection result; wherein the sending position information and the sound source position information are respectively relative position information obtained with corresponding positions as reference points in the two conference rooms;

[0068] Step S04: obtaining the audio and video colocation capability of the audio and video signal transmission between the two conference rooms based on the detection result.

[0069] It should be noted that the two conference rooms for immersive video conferencing are built in accordance with immersive video standard specifications, and the conference room spaces are of the same size or at least the camera collection area spaces are of the same size.

[0070] The method of this embodiment uses a signal generating device in one conference room to automatically send an audio and video test signal, and analyzes the received audio and video signals of the immersive video conferencing system in another conference room, and represents the effect of audio and video colocation with quantitative numerical indicators, thereby solving the problems of strong subjectivity and low efficiency caused by the traditional audio and video colocation test method that relies on manual judgment, and improving the test efficiency and the accuracy of the test results.

[0071] Specifically, in step S01, a signal generating device is used to synchronously send out audio and video signals at intervals during the movement process, and the sending time parameters of each signal are recorded; and the sending position information of each signal when the signal generating device sends out is measured using an infrared signal ranging method.

[0072] The signal generating device may be an audio and video playback device including an audio signal generating module and a video signal generating module, and its movement is achieved through a preset moving track. The movement may be uniform or non-uniform and the generation time of each signal is automatically recorded.

[0073] Exemplarily, the signal generating device sends an audio signal and a video signal synchronously. The audio signal sent by the signal generating device may be a pre-recorded digital "0-9" voice signal, each digital signal voice lasts for 1 second, the interval between two voice signals is 1 second, and a silent packet is played during the interval; the video signal sent by the signal generating device is a digital "0-9" image signal, each image digital signal lasts for 1 second, the interval between two image signals is 1 second, and a black screen image is played during the interval; the start time of recording each audio and video digital signal is t iThe rules for sending audio and video are as follows: Figure 2 shown.

[0074] Specifically, in step S01, an infrared signal distance measurement method is used to measure the distance of the signal generating device. Figure 3 As shown, a three-dimensional coordinate system is established with the reference point at the corner of the conference room where the signal generating device is located, and the coordinates of the signal generating device when each signal is sent are located, and recorded as the sending position information when each signal is sent. i .

[0075] Specifically, in step S01, the video conferencing terminal in the conference room where the signal generating device is located receives the audio signal and the image signal, encapsulates them into a code stream through audio and video coding, and transmits them to the other end, that is, another conference room;

[0076] Specifically, in step S02, in another conference room, after receiving the code stream, the video conferencing terminal decodes and plays the decoded audio and video signals through an audio player and a video player; during detection, speech recognition is performed based on the audio played by the audio player to obtain the corresponding receiving time parameters of each audio signal, and image recognition is performed based on the video played by the video player to obtain the corresponding receiving time parameters of each video signal. Exemplarily, speech recognition is performed based on the audio played by the audio player to obtain each digital voice signal and the corresponding playing time (i.e., the receiving time of the voice signal), which is recorded as S i ; Based on the video played by the video player, image recognition is performed to obtain each digital image signal and the corresponding playback time (i.e., the reception time of the digital signal), which is recorded as V i .like Figure 4 Schematic diagram of the detection and analysis of the audio and video co-location capabilities of immersive video conferencing.

[0077] Specifically, in step S02, a microphone array is used to receive audio, and the sound source is localized by the audio intensity and time difference received by each microphone; specifically, a three-dimensional model is performed on the conference room receiving the audio, such as Figure 5 As shown, a three-dimensional coordinate system is established with the position corresponding to another conference room as the reference point to obtain the three-dimensional coordinates of the sound source position of the received audio, that is, the sound source position information L i .

[0078] If two conference rooms that meet the immersive standard have the same size or the same camera coverage area, then each spatial point in one conference room has a one-to-one correspondence with the spatial point in the other conference room. The sending position information and the sound source position information are the relative position information obtained with the corresponding position as the reference point in the two conference rooms; the relative position information refers to the position in the other conference room that has a corresponding relationship with a certain position in one conference room.

[0079] Specifically, the reference point of the corresponding position here refers to a corner position of the same orientation that has a corresponding relationship with the corner position of the conference room where the signal generating device is located.

[0080] Specifically, in step S03, the sound image parity index includes the mean value U of the sound position deviation value PL PL and variance S PL , the mean value U of the audio and video signal synchronization deviation from PSV PSV and variance S PSV , the mean value of the sound image deviation is U PLSV , respectively, using the following formulas to calculate:

[0081]

[0082]

[0083]

[0084]

[0085] U PLSV =ln(1+U PL )+ln(1+U PSV );

[0086] Among them, i represents the i-th signal emitted by the signal generating device, N represents the total number of signal occurrences; Li is the sound source location information, li is the sending location information; Si is the audio signal receiving time, and Vi is the image signal receiving time.

[0087] Specifically, U PL Indicates the degree of deviation of the sound signal position. A value of 0 indicates no deviation on average. A value greater than 0 indicates that the audio signal has deviated from the position at the receiving end. A larger value indicates a larger deviation and a worse effect.

[0088] Specifically, S. PL It indicates the degree of change of the sound signal position deviation. A value of 0 indicates that the deviation value is stable, and a larger value indicates a greater degree of change.

[0089] Specifically, U PSV Indicates the deviation value of the synchronous playback of audio and video signals at the receiving end. A value of 0 indicates that the audio and video signals are played synchronously at the receiving end without deviation. A larger value indicates a higher degree of asynchrony.

[0090] Specifically, S. PSV Indicates the degree of change in the synchronization of audio and video signals. A value of 0 indicates a smooth change, and a larger value indicates a greater degree of change.

[0091] Specifically, U PLSVIt indicates the average deviation of audio and video parity. A value of 0 indicates the best audio and video parity effect, and a larger value indicates a worse audio and video parity effect.

[0092] Specifically, in step S04, based on U PLSV The value of is used to evaluate the audio and video co-location capability of audio and video signal transmission between two conference rooms. The evaluation criteria are:

[0093] The sound signal deviation value is measured in centimeters, and the audio and video synchronization time is measured in milliseconds:

[0094] 0 PLSV ≤6, the audio-visual synchronization ability is evaluated as high level, indicating that the audio-visual synchronization ability is good;

[0095] 6 PLSV ≤10, the audio-visual synchronization ability is evaluated as medium level, indicating that the audio-visual synchronization ability is acceptable;

[0096] Then U PLSV >10, the audio-visual synchronization ability is evaluated as low level, indicating that the audio-visual synchronization ability is poor.

[0097] Furthermore, based on U PL , S PL , U PSV and S PSV The value of can further locate the audio and video deviation between the two conference rooms.

[0098] Specifically, the sound signal deviation value is measured in centimeters, and the audio and video synchronization time is measured in milliseconds. Each indicator is used to evaluate the audio and video colocation capabilities of the audio and video signal transmission between the two conference rooms as high, medium, and low levels. A high level indicates that the system has a good ability in the corresponding indicator, and a low level indicates a poor ability. For indicators evaluated as low levels, the corresponding problem of the immersive video conferencing system can be located.

[0099] Specifically, the evaluation level thresholds of each indicator are divided into:

[0100] 0 PL ≤20, high level; 20 PL ≤50, intermediate level; U PL >50 low level;

[0101] 0 PSV ≤100, high level; 100 PSV ≤300, medium level; then U PSV >300, low level;

[0102] 0 PL ≤100, high level; 100 PL ​​​​​​​​≤200, medium level; S PL >200, low level

[0103] 0 PSV ≤100, high level; 100 PSV ≤200, medium level; then S PSV >200, low level.

[0104] This embodiment discloses a detection method for testing the audio and video colocation capability of an immersive video conference. By using a signal generating device in a conference room to automatically send an audio and video test signal, the received audio and video signal of the immersive video conferencing system is analyzed in another conference room, and the effect of the audio and video colocation is represented by a quantitative numerical index. This solves the problems of strong subjectivity and low efficiency caused by the traditional audio and video colocation test method that relies on manual judgment, and improves the test efficiency and the accuracy of the test results.

[0105] By using a regular test sequence of audio and video signals and analyzing the reception of multiple signals, the calculated evaluation index values ​​can scientifically and objectively reflect the audio and video colocation capabilities of immersive video conferencing, thereby improving the accuracy of detection and evaluation.

[0106] By calculating the mean value U of the sound position deviation value PL PL and variance S PL , the mean value U of the audio and video signal synchronization deviation from PSV PSV and variance S PSV , the mean value of the sound image deviation is U PLSV , use U PLSV Comprehensive evaluation of audio-visual parity, using U PL , S PL , U PSV , S PSV The calculated result value specifically locates the cause of the audio transmission or video transmission that may cause poor audio-visual co-location capabilities, so that the test results can assist in locating transmission problems in immersive video conferencing.

[0107] System Example

[0108] Another specific embodiment of the present invention discloses a detection system for testing the audio and video co-location capability of immersive video conferencing, such as Figure 6 As shown, it includes: a signal generating end deployed in one conference room and a signal receiving and analyzing end deployed in another conference room.

[0109] Specifically, the signal generating end includes a signal generating device, a position positioning module, a motion trajectory control module and a first information synchronization module.

[0110] ​​Specifically, the signal generating device is used to synchronously send out audio and video signals during the movement and record corresponding sending time parameters.

[0111] Furthermore, the signal generating device includes an audio signal generating module and a video signal generating module; when the signal generating device is moving, the audio signal generating module and the video signal generating module synchronously send out multiple audio and video signals at intervals, and record the sending time parameters of each signal.

[0112] Specifically, the motion trajectory control module is used to control the moving trajectory of the signal generating device; further, for different meeting scenes, the sound source will have different movement modes. Pre-storing a variety of motion trajectories such as vertical, horizontal, inclined, and curved in the signal generating device can improve the accuracy of the test results.

[0113] Specifically, the position positioning module is used to obtain the sending position information when the signal generating device sends out the audio and video signals; optionally, the position positioning module uses an infrared signal ranging method to measure the distance of the signal generating device.

[0114] Specifically, the first information synchronization module is used to synchronize the sending position information, the audio and video signals and the corresponding sending time parameters to the signal receiving and analyzing end; wherein the sending time parameters include the sending time of each audio and video signal as t i , the time when the signal starts to be sent and the time when the signal ends to be sent.

[0115] Specifically, the signal receiving and analyzing end includes a signal detection module, a sound positioning module, a second information synchronization module and a result analysis module.

[0116] Specifically, the signal detection module is used to detect corresponding reception time parameters based on the received audio played by the audio player in the conference room and the received video played by the video player.

[0117] Furthermore, the signal detection module includes an audio signal detection module and a video signal detection module; the audio detection module is used to detect the received audio played from the conference room audio player, perform voice recognition on it, and detect the current sound signal and the reception time of the current sound signal; the video detection module is used to detect the received video played from the conference room video player, perform image recognition on it, and detect the current image signal and the reception time of the current image signal.

[0118] Specifically, the audio location module is used to locate the sound source based on the received audio played by the audio player in the conference room, and obtain the sound source location information of the received audio.

[0119] Furthermore, the audio positioning module uses a microphone array to receive the audio played by the audio player, locates the sound source through the audio intensity and time difference received by each microphone, and obtains the sound source position information of the received audio.

[0120] Specifically, the second information synchronization module is used to receive information sent by the first information synchronization module, including sending location information, audio and video signals sent by the signal sending end, and corresponding sending time parameters.

[0121] Specifically, the result analysis and presentation module is used to calculate the audio and video parity index value to obtain the detection result based on the audio source location information, the receiving time parameter, the sending location information obtained from the second information synchronization module, the audio and video signals sent by the signal sending end, and the corresponding sending time parameter, and evaluate the audio and video parity capability of the audio and video signal transmission between the two conference rooms based on the detection result and present it to the user. The method for calculating the audio and video parity index value and the method for evaluating the audio and video parity capability based on the detection result are the same as the method embodiment.

[0122] like Figure 7 The figure shows the operation flow chart of the detection system showing the audio-visual parity capability.

[0123] This embodiment discloses a detection system for testing the audio and video co-location capability of immersive video conferencing. A motion trajectory control module is introduced at the system signal generating end to control the moving trajectory of the signal generating device, simulate different motion modes of sound sources in different conference scenes, and preset vertical, horizontal, inclined, curved and other diversified motion trajectories of the signal generating device to effectively improve the accuracy of the test results. Compared with the prior art, the other beneficial effects of this embodiment are basically the same as those provided by the method embodiment, and are not described one by one here.

[0124] It should be noted that the above embodiments are based on the same inventive concept and parts not described repeatedly can be used as reference for each other.

[0125] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.

Claims

1. A detection method for testing the audio and video co-location capability of immersive video conferencing, characterized in that: The following steps are involved: In one of the conference rooms, a signal generating device is used to synchronously send out audio and video signals during the movement, and the corresponding sending position information and sending time parameters are recorded; In another conference room, corresponding receiving time parameters are recorded based on the received audio played by the audio player and the received video played by the video player, and sound source positioning is performed based on the received audio to obtain sound source position information of the received audio; Based on the sending position information, the sound source position information, the sending time parameter and the receiving time parameter, the sound and image co-location index value is calculated to obtain the detection result; wherein the sending position information and the sound source position information are the relative position information obtained with the corresponding positions as reference points in the two conference rooms respectively; The audio and video co-location capability of the audio and video signal transmission between the two conference rooms is obtained based on the detection result.

2. The detection method according to claim 1, characterized in that: The sound image parity index includes the mean value U of the sound position deviation value PL PL and variance S PL , the mean value U of the audio and video signal synchronization deviation from PSV PSV and variance S PSV , the mean value of the sound image deviation is U PLSV , respectively, using the following formulas to calculate: U PLSV =ln(1+U PL )+ln(1+U PSV ); Among them, i represents the i-th signal emitted by the signal generating device, N represents the total number of signal occurrences; Li is the sound source location information, li is the sending location information; Si is the audio signal receiving time, and Vi is the image signal receiving time.

3. The detection method according to claim 2, characterized in that: The evaluation of the audio and video co-location capability of the audio and video signal transmission between the two conference rooms based on the detection result includes: the sound signal deviation value is measured in centimeters, the audio and video synchronization time is measured in milliseconds, and when N is 100: 0 PLSV ≤6, the audio-visual synchronization ability is evaluated as high level, indicating that the audio-visual synchronization ability is good;​ 6 PLSV ≤10, the audio-visual synchronization ability is evaluated as medium level, indicating that the audio-visual synchronization ability is acceptable;​ Then U PLSV >10, the audio-visual synchronization ability is evaluated as low level, indicating that the audio-visual synchronization ability is poor.

4. The detection method according to any one of claims 1 to 3, characterized in that: The method of using the signal generating device to synchronously send out audio and video signals and record corresponding first position information and sending time parameters during the movement includes: A signal generating device is used to synchronously send out audio and video signals at intervals during the movement process, and the sending time parameters of each signal are recorded; an infrared signal ranging method is used to measure and obtain the sending position information of each signal sent by the signal generating device.

5. The detection method according to any one of claims 1 to 3, characterized in that: The received audio played by the audio player is received using a microphone array, and the sound source is located through the audio intensity and time difference received by each microphone to obtain the sound source position information of the received audio.

6. A detection system for testing the audio and video co-location capability of immersive video conferencing, characterized in that: include: A signal generating end deployed in one of the conference rooms and a signal receiving and analyzing end deployed in another of the conference rooms; The signal generating end includes a signal generating device, a position positioning module, a motion trajectory control module, and a first information synchronization module; wherein, The signal generating device is used to synchronously send out audio and video signals during the movement and record the corresponding sending time parameters; The motion trajectory control module is used to control the movement trajectory of the signal generating device; The position positioning module is used to obtain the sending position information when the signal generating device sends out the audio and video signals; The first information synchronization module is used to synchronize the sending position information, the audio and video signals and the corresponding sending time parameters to the signal receiving and analyzing end; The signal receiving and analyzing end includes a signal detection module, a sound location identification module, a second information synchronization module and a result analysis module; wherein, The signal detection module is used to detect corresponding reception time parameters based on the received audio played by the audio player in the conference room and the received video played by the video player; The audio location module is used to locate the sound source based on the received audio played by the audio player in the conference room, and obtain the sound source position information of the received audio; The second information synchronization module is used to receive information sent by the first information synchronization module; The result analysis and presentation module is used to calculate the audio and video parity index value to obtain the detection result, evaluate the audio and video parity capability of the audio and video signal transmission between the two conference rooms based on the detection result, and present it to the user.

7. The detection system according to claim 6, characterized in that: The sound and image parity index includes: the mean value U of the sound position deviation value PL PL and variance S PL , the mean value U of the audio and video signal synchronization deviation from PSV PSV and variance S PSV , the mean value of the sound image deviation is U PLSV , respectively, using the following formulas to calculate: U PLSV =ln(1+U PL )+ln(1+U PSV ); Among them, i represents the i-th signal emitted by the signal generating device, N represents the total number of signal occurrences; Li is the sound source location information, li is the sending location information; Si is the audio signal receiving time, and Vi is the image signal receiving time.

8. The detection system according to claim 7, characterized in that: The evaluation of the audio and video co-location capability of the audio and video signal transmission between the two conference rooms based on the detection result includes: the sound signal deviation value is measured in centimeters, the audio and video synchronization time is measured in milliseconds, and when N is 100: 0 PLSV ≤6, the audio-visual synchronization ability is evaluated as high level, indicating that the audio-visual synchronization ability is good;​ 6 PLSV ≤10, the audio-visual synchronization ability is evaluated as medium level, indicating that the audio-visual synchronization ability is acceptable;​ Then U PLSV >10, the audio-visual synchronization ability is evaluated as low level, indicating that the audio-visual synchronization ability is poor.

9. The detection system according to any one of claims 6 to 8, characterized in that: The signal generating device comprises an audio signal generating module and a video signal generating module; The signal generating device synchronously sends out audio and video signals during movement and records corresponding sending time parameters, including: during movement, the audio signal generating module and the video signal generating module synchronously send out multiple audio and video signals at intervals, and record the sending time parameters of each signal.

10. The detection system according to any one of claims 6 to 8, characterized in that: The audio positioning module uses a microphone array to receive the audio played by the audio player, locates the sound source through the audio intensity and time difference received by each microphone, and obtains the sound source position information of the received audio.

Images