Time delay detection method and device, electronic equipment and storage medium

Abstract

The invention provides a time delay detection method and device, electronic equipment and a storage medium, and relates to the technical field of computers. The method comprises the following steps: when it is detected that a current preset time interval is reached, executing video image acquisition to obtain a first image; determining a first serial number and first acquisition time information of the first image, receiving a second serial number and second acquisition time information of a second image sent by the slave control end, the slave control end being used for executing video image acquisition to obtain the second image when it is detected that a current preset time interval is reached, determining a second serial number and second acquisition time information of the second image; and matching the first image with the second image to determine the service time delay corresponding to the current preset time interval. According to the invention, the business time delay is determined based on the interaction mode that the master control end and the slave control end respectively carry out video image acquisition, the actual perception time delay of a user plane can be truly reflected, the accuracy of time delay perception is improved, the time delay error is reduced, and automatic time delay evaluation can be realized.

Description

Background Art

[0002] With the development of computer technology, the application of XR (Extended Reality) technology has become more and more extensive. XR technology can superimpose virtual information to a certain position in the real perspective through device recognition, and through the fusion of real and virtual information, it can realize the interaction of virtual information that is strongly related to the real situation.

[0003] In related technologies, user-side delay measurement mainly relies on manual testing, and automated evaluation has not been achieved. The accuracy of manual measurement is poor, the error is uncertain, and horizontal comparison is impossible. The master and slave ends of the XR service need to be in the same position at the same time to estimate the service delay, and additional cameras are required to assist. Therefore. There is an urgent need for a delay detection method that can be applied to XR services, which can improve the accuracy of delay perception, reduce delay errors, and realize automated delay evaluation.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0005] The present disclosure provides a delay detection method, device, electronic device and storage medium, which at least to a certain extent overcome the problems of poor accuracy of delay perception, large delay error and inability to achieve automated delay evaluation in related technologies.

[0006] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0007] According to one aspect of the present disclosure, a delay detection method is provided, which is executed by a master end, and the master end establishes an XR service connection with a slave end, wherein the master end is used to capture XR video and synchronize it to the slave end, and the slave end is used to play the XR video, including: when detecting that a current preset time interval has been reached, performing video image capture to obtain a first image, the first image being any frame image in the XR video; determining a first sequence number and first acquisition time information of the first image, wherein any frame image in the XR video corresponds to a sequence number; receiving a second sequence number and second acquisition time information of a second image sent by the slave end, wherein the slave end is used to perform video image capture to obtain a second image when detecting that a current preset time interval has been reached, and determining the second sequence number and second acquisition time information of the second image; matching the first image with the second image to determine the service delay corresponding to the current preset time interval.

[0008] In some embodiments, the latency detection method provided by the embodiments of the present disclosure further includes: receiving frame loss information sent from the control end; when the XR video playback is completed, determining the frame loss rate based on the frame loss information.

[0009] In some embodiments, the delay detection method provided by the embodiments of the present disclosure also includes: receiving an asynchronous detection result sent by a slave control end, the slave control end is used to detect the serial number duration corresponding to each frame image in the XR video, and determine the asynchronous detection result according to the serial number duration; determine the delay detection result according to the service delay, frame loss rate and asynchronous detection result corresponding to at least one preset time interval.

[0010] According to another aspect of the present disclosure, a delay detection method is provided, which is executed by a slave end, and the slave end establishes an XR service connection with a master end, wherein the master end is used to capture XR service video and synchronize it to the slave end, and the slave end is used to play the XR service video, including: when detecting that a current preset time interval has been reached, performing video image capture to obtain a second image, the second image being any frame image in the XR video; determining a second sequence number and second acquisition time information of the second image, wherein any frame image in the XR video corresponds to a sequence number; sending the second sequence number and second acquisition time information of the second image to the master end, and the master end is used to perform video image capture to obtain a first image when detecting that a current preset time interval has been reached, the first image being any frame image in the XR video; determining a first sequence number and first acquisition time information of the first image; and matching the first image with the second image to determine the service delay corresponding to the current preset time interval.

[0011] In some embodiments, the delay detection method provided by the embodiments of the present disclosure also includes: determining the mark position corresponding to each frame image in the XR video; determining whether there is a corresponding mark at the serial number position corresponding to each frame image; if the corresponding mark is not detected at the serial number position corresponding to the target frame image, determining the frame loss information of the target frame image, where the target frame image is any frame image in the XR video; sending the frame loss information to the main control end, and the main control end is used to determine the frame loss rate according to the frame loss information when the XR video playback is completed.

[0012] In some embodiments, the delay detection method provided by the embodiments of the present disclosure also includes: detecting the serial number duration corresponding to each frame image in the XR video; determining the asynchronous detection result based on the serial number duration; sending the asynchronous detection result to the main control end, and the main control end is used to determine the delay detection result based on the service delay, frame loss rate and asynchronous detection result corresponding to at least one preset time interval.

[0013] According to another aspect of the present disclosure, a delay detection device is also provided, which is applied to a master control end, and the master control end establishes an XR service connection with a slave control end, wherein the master control end is used to capture XR video and synchronize it to the slave control end, and the slave control end is used to play the XR video, including: an image acquisition module, which is used to perform video image capture to acquire a first image when detecting that a current preset time interval has been reached, and the first image is any frame image in the XR video; an information determination module, which is used to determine a first sequence number and first acquisition time information of the first image, wherein any frame image in the XR video corresponds to a sequence number; an information receiving module, which is used to receive a second sequence number and second acquisition time information of a second image sent by the slave control end, wherein the slave control end is used to perform video image capture to acquire a second image when detecting that a current preset time interval has been reached, and determine the second sequence number and second acquisition time information of the second image; a service delay determination module, which is used to match the first image with the second image to determine the service delay corresponding to the current preset time interval.

[0014] In some embodiments, the delay detection device provided by the embodiments of the present disclosure further includes: a frame loss rate determination module, which is used to receive frame loss information sent from the control end; when the XR video playback is completed, the frame loss rate is determined according to the frame loss information.

[0015] In some embodiments, the delay detection device provided by the embodiments of the present disclosure also includes: an asynchronous detection result receiving module, which is used to receive the asynchronous detection result sent by the slave control end, and the slave control end is used to detect the serial number duration corresponding to each frame image in the XR video, and determine the asynchronous detection result according to the serial number duration; a delay detection result determination module, which is used to determine the delay detection result according to the service delay, frame loss rate and asynchronous detection result corresponding to at least one preset time interval.

[0016] According to another aspect of the present disclosure, a delay detection device is also provided, which is applied to a slave end, and the slave end establishes an XR service connection with a master end, wherein the master end is used to capture XR service video and synchronize it to the slave end, and the slave end is used to play the XR service video, including: an image acquisition module, which is used to perform video image capture to obtain a second image when detecting that a current preset time interval has been reached, and the second image is any frame image in the XR video; an information determination module, which is used to determine a second sequence number and second acquisition time information of the second image, wherein any frame image in the XR video corresponds to a sequence number; an information sending module, which is used to send the second sequence number and second acquisition time information of the second image to the master end, and the master end is used to perform video image capture to obtain a first image when detecting that a current preset time interval has been reached, and the first image is any frame image in the XR video; determine the first sequence number and first acquisition time information of the first image; match the first image with the second image to determine the service delay corresponding to the current preset time interval.

[0017] In some embodiments, the delay detection device provided by the embodiments of the present disclosure also includes: a frame loss information determination module, which is used to determine the mark position corresponding to each frame image in the XR video; determine whether there is a corresponding mark at the serial number position corresponding to each frame image; if the corresponding mark is not detected at the serial number position corresponding to the target frame image, determine the frame loss information of the target frame image, where the target frame image is any frame image in the XR video; send the frame loss information to the main control end, and the main control end is used to determine the frame loss rate according to the frame loss information when the XR video playback is completed.

[0018] In some embodiments, the delay detection device provided by the embodiments of the present disclosure also includes: an asynchronous detection result determination module, which is used to detect the serial number duration corresponding to each frame image in the XR video; determine the asynchronous detection result according to the serial number duration; send the asynchronous detection result to the main control end, and the main control end is used to determine the delay detection result based on the service delay, frame loss rate and asynchronous detection result corresponding to at least one preset time interval.

[0019] According to another aspect of the present disclosure, a delay detection system is also provided, including a master control end and a slave control end, wherein the master control end and the slave control end establish an XR service connection, wherein the master control end is used to capture XR video and synchronize it to the slave control end, and the slave control end is used to play the XR video; the master control end is also used to perform video image capture to obtain a first image when detecting that a current preset time interval has been reached, and the first image is any frame image in the XR video; determine a first sequence number and a first acquisition time information of the first image, wherein any frame image in the XR video corresponds to a sequence number; the slave control end is also used to perform video image capture to obtain a second image when detecting that the current preset time interval has been reached, and the second image is any frame image in the XR video; determine a second sequence number and a second acquisition time information of the second image; send the second sequence number and the second acquisition time information of the second image to the master control end; the master control end is also used to receive the second sequence number and the second acquisition time information of the second image; match the first image with the second image to determine the service delay corresponding to the current preset time interval.

[0020] According to another aspect of the present disclosure, an electronic device is also provided, which includes: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute any one of the above-mentioned delay detection methods by executing the executable instructions.

[0021] According to another aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, any one of the above delay detection methods is implemented.

[0022] According to another aspect of the present disclosure, a computer program product or a computer program is provided, the computer program product or the computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device executes the delay detection method provided in various optional manners in any embodiment of the present disclosure.

[0023] The technical solution provided in the embodiments of the present disclosure determines the service delay based on the interactive mode in which the master end and the slave end respectively perform video image acquisition. It can truly reflect the actual perceived delay of the user surface, improve the accuracy of delay perception, reduce delay error, and realize automated delay evaluation.

[0024] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0026] Figure 1 A schematic diagram showing a system architecture in an embodiment of the present disclosure;

[0027] Figure 2 A flow chart of a delay detection method in an embodiment of the present disclosure is shown;

[0028] Figure 3 Another delay detection method flow chart in an embodiment of the present disclosure is shown;

[0029] Figure 4 A schematic diagram of a delay detection process in an embodiment of the present disclosure is shown;

[0030] Figure 5 A schematic diagram of a delay detection device in an embodiment of the present disclosure is shown;

[0031] Figure 6 A schematic diagram of another delay detection device in an embodiment of the present disclosure is shown;

[0032] Figure 7 A structural block diagram of an electronic device according to an embodiment of the present disclosure is shown;

[0033] Figure 8 A schematic diagram of a computer-readable storage medium in an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0034] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0035] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0036] The specific implementation of the embodiment of the present disclosure is described in detail below with reference to the accompanying drawings.

[0037] Figure 1 FIG. 2 shows an exemplary application system architecture diagram to which the delay detection method in the embodiment of the present disclosure can be applied. Figure 1 As shown, the system architecture 100 may include a master terminal 101 , a network 102 , and a slave terminal 103 .

[0038] Among them, an XR service connection can be established between the master terminal 101 and the slave terminal 103, wherein the master terminal 101 is used to capture the XR video and synchronize it to the slave terminal 103, and the slave terminal 103 is used to play the XR video.

[0039] The main control end 101 can perform video image acquisition to obtain a first image when detecting that the current preset time interval has arrived, and the first image is any frame image in the XR video; determine the first serial number and first acquisition time information of the first image, wherein any frame image in the XR video corresponds to a serial number.

[0040] The slave control end 103 can perform video image acquisition to obtain a second image when detecting that the current preset time interval has been reached, where the second image is any frame image in the XR video; determine the second serial number and second acquisition time information of the second image; and send the second serial number and second acquisition time information of the second image to the master control end 101.

[0041] In addition, the main control end 101 may also receive the second serial number and second acquisition time information of the second image; and match the first image with the second image to determine the service delay corresponding to the current preset time interval.

[0042] The network 102 is a medium for providing a communication link between the master terminal 101 and the slave terminal 103 , and may be a wired network or a wireless network.

[0043] Optionally, the wireless network or wired network described above uses standard communication technology and / or protocol. The network is usually the Internet, but it can also be any network, including but not limited to a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a dedicated network or any combination of a virtual private network). In some embodiments, the data exchanged through the network is represented by technologies and / or formats including Hyper Text Mark-up Language (HTML), Extensible Markup Language (XML), etc. In addition, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPSec) can also be used to encrypt all or some links. In other embodiments, customized and / or dedicated data communication technologies can also be used to replace or supplement the above data communication technologies.

[0044] Exemplarily, the master terminal 101 and the slave terminal 103 may be various electronic devices, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, smart speakers, smart watches, wearable devices, augmented reality devices, virtual reality devices, etc.

[0045] Optionally, the client of the application installed in the master terminal 101 and the slave terminal 103 may be the same, or the client of the same type of application based on different operating systems. Based on the different terminal platforms, the specific form of the client of the application may also be different, for example, the application client may be a mobile client, a PC client, etc.

[0046] Those skilled in the art will know that Figure 1The number of the master terminal 101, the network 102 and the slave terminal 103 is only for illustration, and any number of the master terminal 101, the network 102 and the slave terminal 103 may be provided according to actual needs. The embodiments of the present disclosure are not limited to this.

[0047] Under the above system architecture, a delay detection method is provided in an embodiment of the present disclosure, and the method can be executed by any electronic device with computing and processing capabilities.

[0048] In some embodiments, the delay detection method provided in the embodiments of the present disclosure can be executed by the master end of the above-mentioned system architecture; in other embodiments, the delay detection method provided in the embodiments of the present disclosure can be executed by the slave end 103 in the above-mentioned system architecture; in other embodiments, the delay detection method provided in the embodiments of the present disclosure can be implemented by the master end and the slave end in the above-mentioned system architecture through interaction.

[0049] Figure 2 A flow chart of a delay detection method in an embodiment of the present disclosure is shown as follows: Figure 2 As shown, the delay detection method provided in the embodiment of the present disclosure includes the following steps S202 to S208.

[0050] S202: When the master control end detects that the current preset time interval has arrived, the master control end performs video image acquisition to obtain a first image, where the first image is any frame image in the XR video.

[0051] In some exemplary embodiments, the master end may establish an XR service connection with a slave end, wherein the master end is used to capture XR video and synchronize it to the slave end, and the slave end is used to play the XR video.

[0052] The embodiments of the present disclosure do not limit the application scenarios. By way of example, the application scenarios may be remote conferencing, immersive gaming, emergency rescue, medical assistance, and the like.

[0053] The disclosed embodiments do not limit the content of the XR video. The content of the XR service can be determined according to the application scenario. For example, taking the application scenario as a remote conference, the XR video can be a conference video. In this case, the master control end can capture the conference video and synchronize it to the slave control end. The slave control end can control the playback of the conference video.

[0054] For another example, taking the application scenario of medical rescue as an example, the XR video can be a surgical video. In this case, the master control end can capture the surgical video and synchronize it to the slave control end. The slave control end can control the playback of the surgical video.

[0055] In some possible implementations, the master and slave terminals can pre-install corresponding APPs (Applications), so that the master and slave terminals can be paired based on the APPs to establish an end-to-end XR service connection. In addition, after the master APP is started, the slave APP can be remotely controlled so that the slave APP is automatically started.

[0056] In an exemplary embodiment, the master end may include corresponding configurations of the slave end, so that an end-to-end XR service connection may be established after the master end is started.

[0057] For example, after both the master and slave are started, the master and slave can be time-calibrated to ensure that the time on both ends is consistent. The disclosed embodiment does not limit the accuracy of time, and the accuracy of time can be set based on application scenarios and experience, for example, the accuracy of time can be in seconds or milliseconds.

[0058] In a possible implementation, a stopwatch clock image may be captured on the master end. After the stopwatch is started, the XR viewing device on the master end is aimed at the stopwatch, and attention is paid to focusing on the stopwatch during recording to ensure that the stopwatch time is clear enough for subsequent matching and comparison.

[0059] In an exemplary embodiment, the master and slave terminals can perform XR measurement condition detection. After the XR measurement condition detection starts, terminal status detection, time calibration and synchronization detection can be performed. Then, when the XR measurement condition is met, the XR delay test can be started. If the XR measurement condition is not met, the terminal abnormal module restart, re-time calibration and synchronization operations are performed until the XR measurement condition is met and the XR delay test is started.

[0060] In some embodiments, a preset time interval may be preset, and the master end and the slave end may simultaneously capture video images at each preset time interval. The present disclosure embodiment does not limit the preset time interval, which may be, for example, 0.5 seconds, 1 second, etc.

[0061] Exemplarily, when the main control end detects that the current preset time interval has arrived, the main control end may perform a video image acquisition to obtain the first image.

[0062] In some exemplary embodiments, the video image acquisition may be performed again when the next preset time interval is detected. In the process of the master end acquiring the XR video, the video image acquisition is performed once each time the preset time interval is reached to obtain the corresponding image.

[0063] S204: The main control end determines a first serial number and first acquisition time information of the first image, wherein any frame image in the XR video corresponds to a serial number.

[0064] In some exemplary embodiments, image recognition may be performed on the first image to determine a first serial number corresponding to the first image. Exemplarily, the first serial number may be used to describe a stopwatch clock corresponding to the first image, or the first serial number may be consistent with a frame number corresponding to the first image. In addition, the first acquisition time message may be used to describe the time when the master end acquires the first image.

[0065] It should be noted that after the video image acquisition is performed again at the next preset time interval, the sequence number and acquisition time of the image acquired again can be determined.

[0066] In an exemplary embodiment, if the image cannot be recognized for N consecutive times, an alarm may be issued, where N is a positive integer. Exemplarily, after an alarm appears on the main control end APP, it may be checked whether there are problems with the test conditions and environment.

[0067] S206, the master end receives the second serial number and second acquisition time information of the second image sent by the slave end, wherein the slave end is used to perform video image acquisition to obtain the second image when detecting that the current preset time interval has arrived, and determine the second serial number and second acquisition time information of the second image.

[0068] It should be noted that the method by which the slave end acquires the second image and determines the second serial number and the second acquisition time is consistent with the method by which the master end acquires the first image and determines the first serial number and the first acquisition time, and will not be repeated here.

[0069] In an exemplary embodiment, the master end and the slave end should perform image acquisition at the same time, that is, the first acquisition time information may be consistent with the second acquisition time information.

[0070] S208: The main control end matches the first image with the second image to determine the service delay corresponding to the current preset time interval.

[0071] It should be noted that, since both the master and slave take screenshots at a preset time interval, the first image and the second image should be the same image in an ideal situation. However, due to the time delay, the first image and the second image may be different, so the master can perform image matching to determine the service delay between the master and the slave.

[0072] In a possible implementation, for example, assuming that the first image recognition clock time result is Tk0, the first acquisition time corresponding to the acquisition of the first image at the master end is Tk. The second acquisition time corresponding to the second image matching the first image is T'k, so the service delay = T'k-Tk.

[0073] In the exemplary embodiment, the XR service latency is tested in various scenarios such as different networks and different signal environments. At a certain moment, assuming it is moment k, if the millisecond time obtained by the master and slave end for image acquisition and recognition is consistent, it is considered that the images are matched at this time, and the service latency at a certain moment is obtained by subtracting the first acquisition time and the second acquisition time corresponding to the master and slave ends at this moment.

[0074] In a possible implementation, the average service delay may be calculated based on service delays obtained through multiple calculations. The average service delay may be calculated based on the following formula (1).

[0075]

[0076] In formula (1), L avg It can represent the average delay. k can represent the time, n is the number of times the service delay is calculated, T k It can represent the first acquisition time corresponding to the acquisition of the first image by the master control end, T′ k It can represent the second acquisition time corresponding to the acquisition of the second image by the control end.

[0077] Exemplarily, continuous acquisition of XR video images can be performed on the slave end and the images can be recognized, wherein the frame rate of each frame image can be matched according to the actual frame rate of the service.

[0078] The embodiment of the present disclosure does not limit the counting method of the frame rate. For example, the counter for delay calculation can be used as the frame rate counter, or the corresponding frame rate counter can be adapted according to the actual frame rate of the application to determine the actual frame rate of the service. In the embodiment of the present disclosure, for example, a frame rate counter of 15 per second, 30 per second, 60 per second, or 90 per second can be selected.

[0079] In some exemplary embodiments, the latency detection method provided by the embodiments of the present disclosure may further include: receiving frame loss information sent by the slave control end; when the XR video is played, determining the frame loss rate according to the frame loss information.

[0080] In an exemplary embodiment, the slave end can detect frame loss information in real time during the playback of the XR video and send it to the master end. Afterwards, when the XR video playback is completed, the master end can calculate the frame loss rate based on the frame loss information.

[0081] Exemplarily, any frame loss information may include a sequence number corresponding to a lost frame of image. In addition, the frame loss information may also include a possible frame loss time point of the frame of image.

[0082] Exemplarily, the control end can determine the mark position corresponding to each frame image in the XR video; determine whether there is a corresponding mark at the serial number position corresponding to each frame image; if the corresponding mark is not detected at the serial number position corresponding to the target frame image, determine the frame loss information of the target frame image.

[0083] The embodiments of the present disclosure do not limit the form of the mark. For example, the mark may be a digital mark or a graphic mark. In some embodiments, the mark may be a serial number corresponding to the image.

[0084] In an exemplary embodiment, the slave control end can check the continuity and integrity of the XR video data at the mark position corresponding to each frame image. Check whether there is an expected mark at the mark position corresponding to each frame image. If the mark at the position does not match the expectation, or the corresponding mark is not detected, it is considered that the frame image is lost.

[0085] In an exemplary embodiment, the slave can determine whether each frame is lost based on image processing techniques such as edge detection and feature matching. If data loss is detected, lost_frames is increased by 1. When the XR video playback and recording are completed, the slave can calculate the frame loss rate based on the value of lost_frames and the frame rate counter, and can obtain the corresponding frame loss time point.

[0086] In some exemplary embodiments, the delay detection method provided by the embodiments of the present disclosure may also include: receiving the asynchronous detection result sent by the slave control end, the slave control end being used to detect the serial number duration corresponding to each frame image in the XR video, and determining the asynchronous detection result according to the serial number duration; determining the delay detection result according to the service delay, frame loss rate and the asynchronous detection result corresponding to at least one preset time interval.

[0087] In an exemplary embodiment, the slave control end may perform asynchronous detection during the XR video playback process. For example, the slave control end may detect the duration of each sequence number. If the duration of any sequence number exceeds a preset time threshold, the duration exceeding the preset time threshold may be accumulated until the sequence number changes, and the accumulated duration may be used as the asynchronous detection result.

[0088] Exemplarily, the preset time threshold may be matched according to the frame rate of the XR video. For example, if the sample frame rate is 90 frames per second, then each serial number exists for no more than 1 / 90 second. Therefore, in an embodiment, the preset time threshold may be 1 / 90 second.

[0089] In some embodiments, the delay detection result can be determined according to the following formula (2).

[0090] Q=K1*T1+K2*F+k3*T2 (2)

[0091] Among them, Q can represent the delay detection result, T1 can represent the average service delay, F is the frame loss rate, T2 is the asynchronous detection result, and K1, K2, K3 are corresponding weight coefficients, and the sum of K1, K2, K3 is 1.

[0092] It should be noted that the embodiments of the present disclosure do not limit the values ​​of K1, K2, and K3, and the values ​​of K1, K2, and K3 can be set based on experience or application scenarios.

[0093] The method provided by the embodiment of the present disclosure determines the service delay based on the interactive mode in which the master end and the slave end respectively perform video image acquisition. It can truly reflect the actual perceived delay of the user surface, improve the accuracy of delay perception, reduce delay error, and realize automated delay evaluation.

[0094] Figure 3 Another delay detection method flow chart in the embodiment of the present disclosure is shown as follows: Figure 3 As shown, the delay detection method provided in the embodiment of the present disclosure includes the following steps S302 to S306.

[0095] S302, when the slave control end detects that the current preset time interval has arrived, the slave control end performs video image acquisition to obtain a second image, where the second image is any frame image in the XR video.

[0096] S304, determining a second serial number and second acquisition time information of the second image from the control end, wherein any frame image in the XR video corresponds to a serial number.

[0097] S306, the slave control end sends the second serial number and the second acquisition time information of the second image to the master control end, and the master control end is used to perform video image acquisition to obtain a first image when detecting that the current preset time interval has arrived, where the first image is any frame image in the XR video; determine the first serial number and the first acquisition time information of the first image; match the first image with the second image to determine the service delay corresponding to the current preset time interval.

[0098] In an exemplary embodiment, the delay detection method provided by the embodiment of the present disclosure may also include: determining the mark position corresponding to each frame image in the XR video; determining whether there is a corresponding mark at the serial number position corresponding to each frame image; if the corresponding mark is not detected at the serial number position corresponding to the target frame image, determining the frame loss information of the target frame image, where the target frame image is any frame image in the XR video; sending the frame loss information to the main control end, and the main control end is used to determine the frame loss rate according to the frame loss information when the XR video playback is completed.

[0099] In an exemplary embodiment, the delay detection method provided by the embodiment of the present disclosure may also include: detecting the serial number duration corresponding to each frame image in the XR video; determining the asynchronous detection result according to the serial number duration; sending the asynchronous detection result to the main control end, and the main control end is used to determine the delay detection result according to the service delay, frame loss rate and asynchronous detection result corresponding to at least one preset time interval.

[0100] It should be noted that the implementation of steps S302 to S306 can refer to the corresponding descriptions in the above steps S202 to S208, which will not be repeated here.

[0101] The disclosed embodiment can perform time synchronization and synchronous video image acquisition based on the master and slave ends, and can calculate the XR service delay based on the image recognition comparison results, as well as frame loss, asynchrony, etc. Therefore, the disclosed embodiment can improve the accuracy of delay perception, reduce delay errors, and realize automated delay evaluation.

[0102] It should be noted that the acquisition, storage, use, and processing of data in the technical solution disclosed in this disclosure are in compliance with the relevant provisions of national laws and regulations, and various types of data such as personal identity data, operation data, behavioral data, etc. related to individuals, customers, and groups obtained in the embodiments of this disclosure have all been authorized.

[0103] Figure 4 A schematic diagram of a delay detection process in an embodiment of the present disclosure is shown.

[0104] In this Figure 4 In the test, the master end includes the XR device and the master end APP. The real-time image object of the master end can be based on the millisecond clock and frame rate counter. When the master end starts the test, the master end can trigger the slave end to start. Then, the master end can calibrate the time synchronization with the slave end. Then, the master end can synchronize video image acquisition with the slave end, and perform image recognition and image matching on the acquired images to determine the delay detection result.

[0105] In addition, a stopwatch clock image may be captured at the master end to perform image matching based on the clock image, and the frame rate of the XR video may be determined based on a frame rate counter.

[0106] Based on the same inventive concept, the present disclosure also provides a delay detection device, as described in the following embodiments. Since the principle of solving the problem in the device embodiment is similar to that in the above method embodiment, the implementation of the device embodiment can refer to the implementation of the above method embodiment, and the repeated parts will not be repeated.

[0107] Figure 5 A schematic diagram of a delay detection device in an embodiment of the present disclosure is shown. Figure 5 As shown, the device is applied to the master control end, and the master control end establishes an XR service connection with the slave control end, wherein the master control end is used to collect XR video and synchronize it to the slave control end, and the slave control end is used to play the XR video, including:

[0108] An image acquisition module 501 is used to perform video image acquisition to acquire a first image when detecting that a current preset time interval has been reached, where the first image is any frame of an XR video;

[0109] An information determination module 502 is used to determine a first sequence number and first acquisition time information of a first image, wherein any frame image in the XR video corresponds to a sequence number;

[0110] The information receiving module 503 is used to receive the second serial number and second acquisition time information of the second image sent by the slave control terminal, wherein the slave control terminal is used to perform video image acquisition to obtain the second image and determine the second serial number and second acquisition time information of the second image when detecting that the current preset time interval has arrived;

[0111] The service delay determination module 504 is configured to match the first image with the second image to determine the service delay corresponding to the current preset time interval.

[0112] In some embodiments, the delay detection device provided by the embodiments of the present disclosure further includes:

[0113] The frame loss rate determination module is used to receive the frame loss information sent from the control end; when the XR video is played, the frame loss rate is determined according to the frame loss information.

[0114] In some embodiments, the delay detection device provided by the embodiments of the present disclosure further includes:

[0115] An asynchronous detection result receiving module is used to receive the asynchronous detection result sent by the slave control end, and the slave control end is used to detect the sequence number duration corresponding to each frame image in the XR video, and determine the asynchronous detection result according to the sequence number duration;

[0116] The delay detection result determination module is used to determine the delay detection result according to the service delay, frame loss rate and asynchronous detection result corresponding to at least one preset time interval.

[0117] It should be noted that the above-mentioned image acquisition module 501, information determination module 502, information receiving module 503, and service delay determination module 504 correspond to S202 to S208 in the method embodiment, and the examples and application scenarios implemented by the above-mentioned modules and corresponding steps are the same, but are not limited to the contents disclosed in the above-mentioned method embodiment. It should be noted that the above-mentioned modules as part of the device can be executed in a computer system such as a set of computer executable instructions.

[0118] Figure 6 A schematic diagram of a delay detection device in an embodiment of the present disclosure is shown. Figure 6 As shown, the device is applied to the slave control end, including:

[0119] An image acquisition module 601 is used to perform video image acquisition to acquire a second image when detecting that a current preset time interval has arrived, where the second image is any frame image in the XR video;

[0120] An information determination module 602 is used to determine a second sequence number and second acquisition time information of a second image, wherein any frame image in the XR video corresponds to a sequence number;

[0121] The information sending module 603 is used to send the second serial number and second acquisition time information of the second image to the main control end. The main control end is used to perform video image acquisition to obtain a first image when detecting that the current preset time interval has arrived. The first image is any frame image in the XR video; determine the first serial number and first acquisition time information of the first image; match the first image with the second image to determine the service delay corresponding to the current preset time interval.

[0122] In some embodiments, the delay detection device provided by the embodiments of the present disclosure further includes:

[0123] The frame loss information determination module is used to determine the mark position corresponding to each frame image in the XR video; determine whether there is a corresponding mark at the serial number position corresponding to each frame image; if the corresponding mark is not detected at the serial number position corresponding to the target frame image, determine the frame loss information of the target frame image, and the target frame image is any frame image in the XR video; send the frame loss information to the main control end, and the main control end is used to determine the frame loss rate according to the frame loss information when the XR video playback is completed.

[0124] In some embodiments, the delay detection device provided by the embodiments of the present disclosure further includes:

[0125] The asynchronous detection result determination module is used to detect the serial number duration corresponding to each frame image in the XR video; determine the asynchronous detection result according to the serial number duration; send the asynchronous detection result to the main control end, and the main control end is used to determine the delay detection result according to the service delay, frame loss rate and asynchronous detection result corresponding to at least one preset time interval.

[0126] It should be noted that the above-mentioned image acquisition module 601, information determination module 602, and information sending module 603 correspond to S302 to S306 in the method embodiment, and the examples and application scenarios implemented by the above-mentioned modules and corresponding steps are the same, but are not limited to the contents disclosed in the above-mentioned method embodiment. It should be noted that the above-mentioned modules as part of the device can be executed in a computer system such as a set of computer executable instructions.

[0127] The device provided by the embodiment of the present disclosure determines the service delay based on the interactive mode in which the master end and the slave end respectively perform video image acquisition, which can truly reflect the actual perceived delay of the user surface, improve the accuracy of delay perception, reduce delay error, and realize automated delay evaluation.

[0128] Those skilled in the art will appreciate that various aspects of the present disclosure may be implemented as systems, methods or program products. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as "circuits", "modules" or "systems".

[0129] The embodiment of the present disclosure provides an electronic device, which exemplarily includes: a processor and a memory. The memory can be used to store executable instructions of the processor. The processor is configured to perform the delay detection method provided by the embodiment of the present disclosure by executing the above executable instructions.

[0130] Refer to the following Figure 7 An electronic device 700 according to this embodiment of the present disclosure is described. Figure 7 The electronic device 700 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0131] like Figure 7 As shown, the electronic device 700 is in the form of a general computing device. The components of the electronic device 700 may include but are not limited to: at least one processing unit 710, at least one storage unit 720, and a bus 730 connecting different system components (including the storage unit 720 and the processing unit 710).

[0132] The storage unit stores a program code, which can be executed by the processing unit 710, so that the processing unit 710 performs the steps of various exemplary embodiments of the present disclosure described in the above “Exemplary Method” section of this specification. For example, the processing unit 710 can perform the following steps of the above method embodiment:

[0133] When it is detected that the current preset time interval has been reached, video image acquisition is performed to obtain a first image, where the first image is any frame image in the XR video; a first serial number and first acquisition time information of the first image are determined, where any frame image in the XR video corresponds to a serial number; and a second serial number and second acquisition time information of the second image sent from the control end are received.

[0134] For another example, the processing unit 710 can execute the following steps of the above method embodiment: when detecting that the current preset time interval has arrived, perform video image acquisition to obtain a second image, the second image being any frame image in the XR video; determine a second serial number and second acquisition time information of the second image, wherein any frame image in the XR video corresponds to a serial number; and send the second serial number and second acquisition time information of the second image to the main control end.

[0135] The storage unit 720 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 7201 and / or a cache storage unit 7202 , and may further include a read-only storage unit (ROM) 7203 .

[0136] The storage unit 720 may also include a program / utility 7204 having a set (at least one) of program modules 7205, such program modules 7205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0137] Bus 730 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0138] The electronic device 700 may also communicate with one or more external devices 740 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 700, and / or communicate with any device that enables the electronic device 700 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 750. Furthermore, the electronic device 700 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 760. As shown, the network adapter 760 communicates with other modules of the electronic device 700 via a bus 730. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 700, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0139] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0140] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart may be implemented as a computer program product, which includes: a computer program, which implements the above-mentioned delay detection method when executed by a processor.

[0141] In an exemplary embodiment of the present disclosure, a computer-readable storage medium is also provided, on which a computer program is stored, and when the computer program is executed by a processor, the delay detection method provided in the embodiment of the present disclosure can be implemented. The computer-readable storage medium can be a readable signal medium or a readable storage medium.

[0142] Figure 8 A schematic diagram of a computer-readable storage medium in an embodiment of the present disclosure is shown. Figure 8As shown, a program product capable of implementing the above method of the present disclosure is stored on the computer-readable storage medium 800. In some possible implementations, various aspects of the present disclosure may also be implemented in the form of a program product, which includes a program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary implementations of the present disclosure described in the above “Exemplary Method” section of this specification.

[0143] More specific examples of computer-readable storage media in the present disclosure may include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0144] In the present disclosure, a computer readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, wherein a readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A readable signal medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0145] Alternatively, the program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0146] In a specific implementation, the program code for performing the operation of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0147] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0148] In addition, although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0149] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0150] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The description and examples are to be regarded as exemplary only, and the true scope of the present disclosure is indicated by the appended claims.

Claims

1. A delay detection method, characterized in that: The master control terminal is executed to establish an extended reality XR service connection with the slave control terminal, wherein the master control terminal is used to collect XR video and synchronize it to the slave control terminal, and the slave control terminal is used to play the XR video, including: When the detection reaches the current preset time interval, perform video image acquisition to obtain a first image, where the first image is any frame image in the XR video; Determine a first sequence number and first acquisition time information of the first image, wherein any frame image in the XR video corresponds to a sequence number; receiving a second sequence number and second acquisition time information of a second image sent by the slave control end, wherein the slave control end is used to perform video image acquisition to obtain a second image when detecting that the current preset time interval has arrived, and determine a second sequence number and second acquisition time information of the second image; The first image is matched with the second image to determine a service delay corresponding to the current preset time interval.

2. The delay detection method according to claim 1, characterized in that: The method further comprises: Receiving frame loss information sent by the slave control end; When the XR video is played, the frame loss rate is determined according to the frame loss information.

3. The delay detection method according to claim 1 or 2, characterized in that: The method further comprises: receiving an asynchronous detection result sent by the slave control end, wherein the slave control end is used to detect a sequence number duration corresponding to each frame image in the XR video, and determine the asynchronous detection result according to the sequence number duration; Determine the delay detection result according to the service delay, frame loss rate and the asynchronous detection result corresponding to at least one preset time interval.

4. A delay detection method, characterized in that: The slave control terminal is executed to establish an XR service connection with the master control terminal, wherein the master control terminal is used to collect XR service video and synchronize it to the slave control terminal, and the slave control terminal is used to play the XR service video, including: When the detection reaches the current preset time interval, performing video image acquisition to obtain a second image, where the second image is any frame image in the XR video; Determine a second sequence number and second acquisition time information of the second image, wherein any frame image in the XR video corresponds to a sequence number; The second serial number and second acquisition time information of the second image are sent to the main control end, and the main control end is used to perform video image acquisition to obtain a first image when detecting that a current preset time interval has arrived, where the first image is any frame image in the XR video; determine the first serial number and first acquisition time information of the first image; match the first image with the second image to determine the service delay corresponding to the current preset time interval.

5. The delay detection method according to claim 4, characterized in that: The method further comprises: Determine a marker position corresponding to each frame image in the XR video; Determine whether there is a corresponding mark at the serial number position corresponding to each frame image; If a corresponding mark is not detected at the serial number position corresponding to the target frame image, frame loss information of the target frame image is determined, and the target frame image is any frame image in the XR video; The frame loss information is sent to the master control end, and the master control end is used to determine the frame loss rate according to the frame loss information when the XR video is played.

6. The delay detection method according to claim 4 or 5, characterized in that: The method further comprises: Detecting the duration of the sequence number corresponding to each frame image in the XR video; Determine the asynchronous detection result according to the sequence number duration; The asynchronous detection result is sent to the master control end, and the master control end is used to determine the delay detection result according to the service delay and frame loss rate corresponding to at least one preset time interval and the asynchronous detection result.

7. A delay detection system, characterized in that: It includes a master control end and a slave control end, wherein the master control end and the slave control end establish an XR service connection, wherein the master control end is used to collect XR video and synchronize it to the slave control end, and the slave control end is used to play the XR video; The main control end is further used to perform video image acquisition to obtain a first image when detecting that a current preset time interval has arrived, where the first image is any frame image in the XR video; determine a first sequence number and first acquisition time information of the first image, where any frame image in the XR video corresponds to a sequence number; The slave control end is further configured to perform video image acquisition to obtain a second image when detecting that the current preset time interval has arrived, where the second image is any frame of the XR video; determine a second sequence number and second acquisition time information of the second image; and send the second sequence number and second acquisition time information of the second image to the master control end; The main control end is further used to receive the second sequence number and second acquisition time information of the second image; and match the first image with the second image to determine the service delay corresponding to the current preset time interval.

8. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to execute the delay detection method according to any one of claims 1 to 6 by executing the executable instructions.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the delay detection method according to any one of claims 1 to 6 is implemented.

10. A computer program product, characterized in that The computer program product includes computer instructions, which are stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device executes the delay detection method as described in any one of claims 1 to 6.

Images