Operation time delay test method, device and equipment of master-slave system and medium

Abstract

The invention provides an operation time delay testing method, device and equipment of a master-slave system and a medium, and the method comprises the steps: carrying out the color space conversion of a plurality of video frames of the master-slave system in an operation process, determining a target video frame, and carrying out the detection of a color threshold range of a target based on the plurality of target video frames, determining outer contours of all shapes in each target video; traversing the outer contour of each shape, and determining the contour of the detection target of each target video frame and the center coordinate of the detection target; for each target video frame, determining a two-dimensional coordinate offset of the detection target of the frame based on the center coordinate of the detection target of the frame and the initial center coordinate of the detection target of the first target video frame; and based on the two-dimensional coordinate offset of the detection target of each target video frame, determining master-slave operation time delays of the master-slave system in different stages. And the delay between the master and the slave can be quickly and accurately calculated, so that the visual measurement and evaluation of the time delay can be realized.

Description

Technical Field

[0001] The present application relates to the technical field of master-slave system control, and in particular to a method, device, equipment and medium for testing the operation delay of a master-slave system. Background Art

[0002] Currently, most methods for testing master-slave operation delays rely on various sensors (such as displacement sensors, gyroscopes, acceleration sensors, etc.). These sensors detect the motion state of the master end (such as the controller joystick) and the slave end (such as the end of the robot), record their displacement changes, and calculate the time difference to derive the master-slave delay. Although existing methods can measure the total master-slave delay, they cannot break down the source of the delay. For example, in a complete master-slave control system, the delay may come from multiple links: the time it takes for the master controller to capture the action intention, the time it takes for data to be transmitted to the main control computer, the time it takes for the main control computer to execute the algorithm, and the time it takes to drive the robot arm to actually move. Therefore, how to accurately determine the master-slave operation delay has become a technical issue that cannot be underestimated. Summary of the Invention

[0003] In view of this, the purpose of this application is to provide a method, device, equipment and medium for testing the operation delay of a master-slave system. By setting the two-dimensional coordinate offset of the detection target at the master and slave ends during the movement, the delay between the master and slave can be quickly and accurately calculated, thereby realizing visual measurement and evaluation of the delay.

[0004] An embodiment of the present application provides a method for testing operation delay of a master-slave system, the method comprising:

[0005] Performing color space conversion on multiple video frames during operation of the master-slave system to determine a target video frame, and determining the outer contours of all shapes in each target video based on the multiple target video frames and a color threshold range of a detection target; wherein a first detection target among the detection targets is set at a master end of the master-slave system, and a second detection target is set at a slave end of the master-slave system;

[0006] Traversing the outer contour of each shape to determine the contour of the detection target and the center coordinates of the detection target in each target video frame;

[0007] For each target video frame, determining a two-dimensional coordinate offset of the detection target in the frame based on the center coordinates of the detection target in the frame and the initial center coordinates of the detection target in the first target video frame;

[0008] Based on the two-dimensional coordinate offset of the detection target in each target video frame, the master-slave operation delay of the master-slave system in different stages is determined.

[0009] In a possible implementation manner, determining the outer contours of all shapes in each target video based on the plurality of target video frames and the color threshold range of the detection target includes:

[0010] Extracting color regions from the plurality of target video frames based on the color threshold range, and generating a mask image based on the color regions;

[0011] Performing binarization processing, noise removal processing, and hole filling processing on the mask image of each target video frame to determine a binary image of each target video frame;

[0012] The outer contours of all shapes are extracted from the binary image of each target video frame.

[0013] In a possible implementation manner, traversing the outer contour of each shape to determine the contour of the detection target and the center coordinates of the detection target in each target video frame includes:

[0014] Detecting whether the area of ​​the region corresponding to the outer contour of the shape is within the area range of the region corresponding to the detection target;

[0015] If not, skip the outer contour of this shape and execute the outer contour of the next shape;

[0016] If so, detecting whether the equivalent radius of the outer contour of the shape is within the equivalent radius range corresponding to the detection target;

[0017] If it is within the equivalent radius range, the outer contour of the shape is taken as a candidate outer contour; if it is not within the equivalent radius range, the outer contour of the shape is skipped and the outer contour of the next shape is executed;

[0018] Check whether the axis ratio and the number of corner points of the candidate outer contour meet the requirements of the detection target. If they do, determine the candidate outer contour as the candidate outer contour and determine the center coordinates.

[0019] In one possible implementation, for each target video frame, determining a two-dimensional coordinate offset of the detected target in the frame based on the center coordinates of the detected target in the frame and the initial center coordinates of the detected target in the first target video frame includes:

[0020] Determining a two-dimensional coordinate offset of the first detection target in the frame based on the center coordinates of the first detection target in the frame and the initial center coordinates of the first detection target;

[0021] Based on the center coordinates of the second detection target in the frame and the initial center coordinates of the second detection target, a two-dimensional coordinate offset of the second detection target in the frame is determined.

[0022] In one possible implementation, determining the master-slave operation delay of the master-slave system at different stages based on the two-dimensional coordinate offset of the detection target in each target video frame includes:

[0023] Constructing a three-dimensional trajectory graph based on the changes in the two-dimensional coordinate offset of the first detection target and the offset of the second detection target in each target video frame over time;

[0024] Based on the time points at which the two-dimensional coordinate offset of the first detection target changes significantly and the time points at which the two-dimensional coordinate offset of the second detection target changes significantly in the three-dimensional trajectory diagram, the master-slave operation delays of the master-slave system at different stages are determined.

[0025] In one possible implementation, determining the master-slave operation delay of the master-slave system at different stages based on the time point at which the two-dimensional coordinate offset of the first detection target and the time point at which the two-dimensional coordinate offset of the second detection target in the three-dimensional trajectory graph significantly changes includes:

[0026] Determining a first master-slave operation delay of the master-slave system based on a time point at which the two-dimensional coordinate offset of the first detection target in the three-dimensional trajectory graph first significantly changes and a time point at which the two-dimensional coordinate offset of the second detection target first significantly changes;

[0027] The master-slave operation delays of the master-slave system at different stages are further determined based on the time point of the next significant change of the first detection target and the time point of the next significant change of the second detection target.

[0028] In a possible implementation manner, the detection targets are a circular mark and a square mark, and the shapes of the first detection target and the second detection target are consistent or inconsistent.

[0029] The embodiment of the present application further provides a master-slave operation delay test device of a master-slave system, the master-slave operation delay test device comprising:

[0030] a video frame processing module, configured to perform color space conversion on multiple video frames during operation of the master-slave system to determine a target video frame, and determine the outer contours of all shapes in each target video based on the multiple target video frames and the color threshold range of the detection target; wherein a first detection target among the detection targets is set at the master end of the master-slave system, and a second detection target is set at the slave end of the master-slave system;

[0031] A target contour determination module is used to traverse the outer contour of each shape to determine the contour of the detection target and the center coordinates of the detection target in each target video frame;

[0032] an offset determination module, configured to determine, for each target video frame, a two-dimensional coordinate offset of the detection target in the frame based on the center coordinates of the detection target in the frame and the initial center coordinates of the detection target in the first target video frame;

[0033] The time delay determination module is used to determine the master-slave operation time delay of the master-slave system at different stages based on the two-dimensional coordinate offset of the detection target in each target video frame.

[0034] An embodiment of the present application also provides an electronic device, comprising: a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the steps of the above-mentioned master-slave system operation delay testing method are performed.

[0035] An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned method for testing the operation delay of the master-slave system are executed.

[0036] An embodiment of the present application provides an operation delay testing method, apparatus, device and medium for a master-slave system. The operation delay testing method includes: performing color space conversion on multiple video frames during the operation of the master-slave system to determine a target video frame, and determining the outer contours of all shapes in each target video based on the multiple target video frames and the color threshold range of the detection target; wherein the first detection target among the detection targets is set at the master end of the master-slave system, and the second detection target is set at the slave end of the master-slave system; traversing the outer contour of each shape to determine the contour of the detection target and the center coordinates of the detection target in each target video frame; for each target video frame, determining the two-dimensional coordinate offset of the detection target in the frame based on the center coordinates of the detection target in the frame and the initial center coordinates of the detection target in the first target video frame; and determining the master-slave operation delay of the master-slave system at different stages based on the two-dimensional coordinate offset of the detection target in each target video frame. By setting the two-dimensional coordinate offset of the detection target at the master and slave ends during the movement, the delay between the master and slave can be quickly and accurately calculated, thereby achieving visual measurement and evaluation of the delay.

[0037] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0039] Figure 1 This is one of the flow charts of a method for testing the operation delay of a master-slave system provided in an embodiment of the present application;

[0040] Figure 2 This is a second flow chart of a method for testing the operation delay of a master-slave system provided in an embodiment of the present application;

[0041] Figure 3 A schematic diagram of the structure of a master-slave operation delay test device for a master-slave system provided in an embodiment of the present application;

[0042] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application for which protection is claimed, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, each other embodiment obtained by those skilled in the art without making creative work falls within the scope of protection of the present application.

[0044] First, the application scenarios to which this application is applicable are introduced. This application can be applied in the field of master-slave system control technology.

[0045] Research has found that the current methods for testing master-slave operation delays mostly rely on various sensors (such as displacement sensors, gyroscopes, acceleration sensors, etc.). These sensors detect the motion state of the master end (such as the controller joystick) and the slave end (such as the slave end of the master-slave system), record their displacement changes, and calculate the time difference to obtain the master-slave delay. Although existing methods can measure the total master-slave delay, they cannot break down the source of the delay. For example, in a complete master-slave control system, the delay may come from multiple links: the time it takes for the master controller to capture the action intention, the time it takes for the data to be transmitted to the main control computer, the time it takes for the main control computer to execute the algorithm, and the time it takes to drive the actual movement of the robotic arm. Therefore, how to accurately determine the master-slave operation delay has become a technical issue that cannot be underestimated.

[0046] Based on this, an embodiment of the present application provides an operation delay testing method for a master-slave system. By setting the two-dimensional coordinate offset of the detection target at the master and slave ends during the movement, the delay between the master and slave is quickly and accurately calculated, thereby realizing visual measurement and evaluation of the delay.

[0047] See also Figure 1 , Figure 1 This is one of the flow charts of a method for testing the operation delay of a master-slave system provided in an embodiment of the present application. Figure 1 As shown in , the operation delay testing method provided by the embodiment of the present application includes:

[0048] S101: performing color space conversion on multiple video frames during operation of the master-slave system to determine a target video frame, and determining the outer contours of all shapes in each target video based on the multiple target video frames and a color threshold range of the detection target.

[0049] In this step, multiple video frames of the master-slave system during operation are converted from RGB color space to HSV color space, and the outer contours of all shapes in each target video are determined based on multiple target video frames and the color threshold range of the detection target.

[0050] The color threshold range of the detection target is pre-set. For example, if the detection target is a circular mark, the color region of the circular mark is within the color threshold range of LOWER_RB and HIGHT_RB in the HSV color space.

[0051] Here, the first detection target among the detection targets is set at the master end of the master-slave system, and the second detection target is set at the slave end of the master-slave system; the detection targets are circular marks and square marks, and the shapes of the first detection target and the second detection target are consistent or inconsistent.

[0052] The detection target may also be a mark of other shapes, which is not specifically limited in this part.

[0053] In a possible implementation manner, determining the outer contours of all shapes in each target video based on the plurality of target video frames and the color threshold range of the detection target includes:

[0054] A: extracting color regions from the plurality of target video frames based on the color threshold range, and generating a mask image based on the color regions.

[0055] Here, the color area corresponding to the color threshold range is extracted from the HSV images in the multiple target video frames to generate a mask image.

[0056] B: performing binarization processing, noise removal processing, and hole filling processing on the mask image of each target video frame to determine the binary image of each target video frame; extracting the outer contours of all shapes in the binary image of each target video frame.

[0057] Here, the mask image is binarized, and morphological closing and opening operations are used to remove noise and fill holes, thereby optimizing the contour extraction effect, thereby obtaining an accurately changed binary image and extracting the outer contours of all shapes in the binary image.

[0058] S102: Traverse the outer contour of each shape to determine the contour of the detection target and the center coordinates of the detection target in each target video frame.

[0059] In this step, the outer contour of each shape is traversed to determine the contour and center coordinates of the detection target in each target video frame.

[0060] In a possible implementation manner, traversing the outer contour of each shape to determine the contour of the detection target and the center coordinates of the detection target in each target video frame includes:

[0061] (1) Detect whether the area of ​​the region corresponding to the outer contour of the shape is within the area range of the region corresponding to the detection target.

[0062] Here, if the detection target is a circular mark, the area range of the region corresponding to the detection target is determined in advance based on the area of ​​the region composed of the minimum radius and the maximum radius required for circular mark detection.

[0063] (2): If not, skip the outer contour of the shape and execute the outer contour of the next shape; if so, detect whether the equivalent radius of the outer contour of the shape is within the equivalent radius range corresponding to the detection target.

[0064] Here, the equivalent radius is determined based on the perimeter of the outer contour of the shape. If the detection target is a circular mark, the equivalent radius range corresponding to the detection target is determined based on the minimum radius and the maximum radius.

[0065] (3): If it is within the equivalent radius range, the outer contour of the shape is taken as a candidate outer contour; if it is not within the equivalent radius range, the outer contour of the shape is skipped and the outer contour of the next shape is executed.

[0066] (4): Detect whether the axis ratio and the number of corner points of the candidate outer contour meet the requirements of the detection target. If so, determine the candidate outer contour as the candidate outer contour and determine the center coordinates.

[0067] Here, for the circular mark as the detection target, the current candidate outer contour is fitted using the ellipse fitting method, and the lengths of its major and minor axes and the coordinates of the center point are obtained. The axis ratio is further calculated. If the axis ratio is within the axis ratio range of the detection target (0.8-1.1), the candidate outer contour is close to a circle. Polygonal approximation is then performed on the candidate outer contour, and the number of corner points after approximation is counted. If the number of corner points is greater than 4, it means that the shape of the candidate outer contour is closer to a circle than a triangle or rectangle. The candidate outer contour is then determined as the candidate outer contour and the center coordinates are determined.

[0068] The requirements for the detection target include the axis ratio requirement and the number of corner points. The axis ratio requirements and the number of corner points required for detection targets of different shapes are different and are not specifically limited in this part.

[0069] Here, the identities of the master and slave are determined based on the positional relationship of the center coordinates in the X-axis direction of the image. For example, if the master is on the left side of the image and the slave is on the right side, then their center coordinates X in the image should satisfy X 主机 <X 从机 .

[0070] S103: For each target video frame, based on the center coordinates of the detection target in the frame and the initial center coordinates of the detection target in the first target video frame, determine a two-dimensional coordinate offset of the detection target in the frame.

[0071] In this step, the two-dimensional coordinate offset of the detection target in the frame is determined based on the center coordinates of the detection target in the frame and the initial center coordinates of the detection target in the first target video frame.

[0072] In one possible implementation, for each target video frame, determining a two-dimensional coordinate offset of the detected target in the frame based on the center coordinates of the detected target in the frame and the initial center coordinates of the detected target in the first target video frame includes:

[0073] Based on the center coordinates of the first detection target in the frame and the initial center coordinates of the first detection target, the two-dimensional coordinate offset of the first detection target in the frame is determined; based on the center coordinates of the second detection target in the frame and the initial center coordinates of the second detection target, the two-dimensional coordinate offset of the second detection target in the frame is determined.

[0074] Here, after all frame images are analyzed, the center coordinates of the first detection target and the center coordinates of the second detection target in each frame are obtained. The initial center coordinates of the detection target in the first target frame are used as the reference origin for each, and the relative offset calculation for subsequent frames is performed using the following formula:

[0075] △x=X 当前帧 -X 第一帧

[0076] △y=Y 当前帧 -Y 第一帧

[0077] Where △x is the horizontal offset of the center coordinate, and △y is the vertical offset of the center coordinate. Here, the initial center coordinate of the first detection target is the initial center coordinate of the first detection target in the initial frame, and the initial center coordinate of the second detection target is the initial center coordinate of the second detection target in the initial frame.

[0078] It should be noted that since the first detection target is set on the master end and the second detection target is set on the slave end, the two-dimensional coordinate offset of the first detection target is the two-dimensional coordinate offset of the master end, and the two-dimensional coordinate offset of the second detection target is the two-dimensional coordinate offset of the slave end, that is, the master-slave operation delay of the master-slave system at different stages is subsequently determined based on the two-dimensional coordinate offset of the detection target.

[0079] S104: Determine the master-slave operation delay of the master-slave system at different stages based on the two-dimensional coordinate offset of the detection target in each target video frame.

[0080] In this step, the master-slave operation delays of the master-slave system at different stages are determined according to the two-dimensional coordinate offset of the detection target in each target video frame.

[0081] In one possible implementation, determining the master-slave operation delay of the master-slave system at different stages based on the two-dimensional coordinate offset of the detection target in each target video frame includes:

[0082] a: constructing a three-dimensional trajectory graph based on the changes of the two-dimensional coordinate offset of the first detection target and the offset of the second detection target in each target video frame over time.

[0083] Here, after calculating the 2D coordinate offsets (△x, y) of the first and second detection targets in each frame, a 3D trajectory graph can be constructed to visually display their motion trends. The horizontal axis (X-axis) represents the horizontal offset △x of the center coordinates; the vertical axis (Y-axis) represents the frame number, indicating the chronological order; and the vertical axis (Z-axis) represents the vertical offset △y of the center coordinates.

[0084] b: Based on the time points at which the two-dimensional coordinate offset of the first detection target changes significantly and the time points at which the two-dimensional coordinate offset of the second detection target changes significantly in the three-dimensional trajectory diagram, determine the master-slave operation delay of the master-slave system at different stages.

[0085] Here, the master-slave operation delay of the master-slave system at different stages is determined according to the time point when the two-dimensional coordinate offset of the first detection target and the time point when the two-dimensional coordinate offset of the second detection target change significantly in the three-dimensional trajectory diagram.

[0086] In one possible implementation, determining the master-slave operation delay of the master-slave system at different stages based on the time point at which the two-dimensional coordinate offset of the first detection target and the time point at which the two-dimensional coordinate offset of the second detection target in the three-dimensional trajectory graph significantly changes includes:

[0087] I: Based on the time point when the two-dimensional coordinate offset of the first detection target in the three-dimensional trajectory diagram first changes significantly and the time point when the two-dimensional coordinate offset of the second detection target first changes significantly, determine the first master-slave operation delay of the master-slave system.

[0088] Here, the first master-slave operation delay of the master-slave system is determined based on the time point when the two-dimensional coordinate offset of the first detection target in the three-dimensional trajectory diagram first changes significantly and the time point when the two-dimensional coordinate offset of the second detection target first changes significantly.

[0089] II: Continue to determine the master-slave operation delay of the master-slave system at different stages based on the time point of the next significant change of the first detection target and the time point of the next significant change of the second detection target.

[0090] Here, a time point of a significant change corresponds to a different stage, which may be a data transmission stage, a task execution stage, or other stages.

[0091] In this application, a distinctively colored indicator light is fixed to the master-side operating lever, an indicator light (detection target) is installed on the main control motherboard, and a motherboard program is used to illuminate the indicator light for several milliseconds whenever the motherboard sends a message to the main control computer. When the main control computer receives a signal from the main controller motherboard, the circle displayed on the screen is immediately moved. After the main controller completes the motion control algorithm, it moves the circle on the screen back to its original position. A distinctively colored circle is fixed to the slave end of the robotic arm. The time delay between different stages of master-slave control can then be calculated based on actual needs by detecting the relative displacement of different circles and the relationship between the brightness of the indicator light and time.

[0092] For further information, see Figure 2 , Figure 2 This is a second flow chart of a method for testing the operation delay of a master-slave system provided in an embodiment of the present application. Figure 2 The implementation method shown in the embodiment takes the detection target as a circle as an example: input video frame, perform color space conversion on the video frame, generate a mask image after conversion to HSV color space, perform binarization, morphological processing, and contour extraction on the mask image to obtain the outer contour of each shape in the video frame, filter the outer contours of all shapes to filter the contours of approximate circles, sort the centers of the circles according to the X coordinates, detect whether the video frame is the last frame, if not, continue to execute the above steps, and if so, determine the two-dimensional coordinate offset of the detection target of each frame and perform offset visualization processing to determine the master-slave operation delay of the master-slave system.

[0093] The embodiment of the present application provides an operation delay test method for a master-slave system, the operation delay test method comprising: performing color space conversion on multiple video frames of the master-slave system during operation to determine a target video frame, and determining the outer contours of all shapes in each target video based on the color threshold range of the multiple target video frames and the detection target; wherein the first detection target among the detection targets is set at the master end of the master-slave system, and the second detection target is set at the slave end of the master-slave system; traversing the outer contour of each shape to determine the contour of the detection target and the center coordinates of the detection target in each target video frame; for each target video frame, based on the center coordinates of the detection target of the frame and the initial center coordinates of the detection target of the first target video frame, determining the two-dimensional coordinate offset of the detection target of the frame; based on the two-dimensional coordinate offset of the detection target of each target video frame, determining the master-slave operation delay of the master-slave system at different stages. By setting the two-dimensional coordinate offset of the detection target at the master end and the slave end during the movement process, the delay between the master and the slave is quickly and accurately calculated, thereby realizing visual measurement and evaluation of the delay.

[0094] See also Figure 3 , Figure 3This is a schematic diagram of a master-slave operation delay test device for a master-slave system provided in an embodiment of the present application. Figure 3 As shown in , the master-slave operation delay testing device 300 includes:

[0095] The video frame processing module 310 is configured to perform color space conversion on multiple video frames during operation of the master-slave system to determine a target video frame, and determine the outer contours of all shapes in each target video based on the multiple target video frames and the color threshold range of the detection target; wherein a first detection target among the detection targets is set at the master end of the master-slave system, and a second detection target is set at the slave end of the master-slave system;

[0096] The target contour determination module 320 is configured to traverse the outer contour of each shape to determine the contour of the detection target and the center coordinates of the detection target in each target video frame;

[0097] an offset determination module 330 for determining, for each target video frame, a two-dimensional coordinate offset of the detection target in the frame based on the center coordinates of the detection target in the frame and the initial center coordinates of the detection target in the first target video frame;

[0098] The delay determination module 340 is configured to determine the master-slave operation delay of the master-slave system at different stages based on the two-dimensional coordinate offset of the detection target in each target video frame.

[0099] Furthermore, when the video frame processing module 310 is used to determine the outer contours of all shapes in each target video based on the plurality of target video frames and the color threshold range of the detection target, the video frame processing module 310 is specifically used to:

[0100] Extracting color regions from the plurality of target video frames based on the color threshold range, and generating a mask image based on the color regions;

[0101] Performing binarization processing, noise removal processing, and hole filling processing on the mask image of each target video frame to determine a binary image of each target video frame;

[0102] The outer contours of all shapes are extracted from the binary image of each target video frame.

[0103] Furthermore, when the target contour determination module 320 is used to traverse the outer contour of each shape and determine the contour of the detection target and the center coordinates of the detection target in each target video frame, the target contour determination module 320 is specifically used to:

[0104] Detecting whether the area of ​​the region corresponding to the outer contour of the shape is within the area range of the region corresponding to the detection target;

[0105] If not, skip the outer contour of this shape and execute the outer contour of the next shape;

[0106] If so, detecting whether the equivalent radius of the outer contour of the shape is within the equivalent radius range corresponding to the detection target;

[0107] If it is within the equivalent radius range, the outer contour of the shape is taken as a candidate outer contour; if it is not within the equivalent radius range, the outer contour of the shape is skipped and the outer contour of the next shape is executed;

[0108] Check whether the axis ratio and the number of corner points of the candidate outer contour meet the requirements of the detection target. If they do, determine the candidate outer contour as the candidate outer contour and determine the center coordinates.

[0109] Furthermore, when the offset determination module 330 is used to determine, for each target video frame, the two-dimensional coordinate offset of the detection target in the frame based on the center coordinates of the detection target in the frame and the initial center coordinates of the detection target in the first target video frame, the offset determination module 330 is specifically configured to:

[0110] Determining a two-dimensional coordinate offset of the first detection target in the frame based on the center coordinates of the first detection target in the frame and the initial center coordinates of the first detection target;

[0111] Based on the center coordinates of the second detection target in the frame and the initial center coordinates of the second detection target, a two-dimensional coordinate offset of the second detection target in the frame is determined.

[0112] Furthermore, the delay determination module 340 is used to determine the master-slave operation delay of the master-slave system at different stages based on the two-dimensional coordinate offset of the detected target in each target video frame. The delay determination module 340 is specifically used to:

[0113] Constructing a three-dimensional trajectory graph based on the changes in the two-dimensional coordinate offset of the first detection target and the offset of the second detection target in each target video frame over time;

[0114] Based on the time points at which the two-dimensional coordinate offset of the first detection target changes significantly and the time points at which the two-dimensional coordinate offset of the second detection target changes significantly in the three-dimensional trajectory diagram, the master-slave operation delays of the master-slave system at different stages are determined.

[0115] Furthermore, the delay determination module 340 is used to determine the master-slave operation delay of the master-slave system at different stages based on the time point at which the two-dimensional coordinate offset of the first detection target in the three-dimensional trajectory graph changes significantly and the time point at which the two-dimensional coordinate offset of the second detection target changes significantly. The delay determination module 340 is specifically used to:

[0116] Determining a first master-slave operation delay of the master-slave system based on a time point at which the two-dimensional coordinate offset of the first detection target in the three-dimensional trajectory graph first significantly changes and a time point at which the two-dimensional coordinate offset of the second detection target first significantly changes;

[0117] The master-slave operation delays of the master-slave system at different stages are further determined based on the time point of the next significant change of the first detection target and the time point of the next significant change of the second detection target.

[0118] An embodiment of the present application provides a master-slave operation delay testing device for a master-slave system, the master-slave operation delay testing device comprising: a video frame processing module, configured to perform color space conversion on multiple video frames during operation of the master-slave system to determine a target video frame, and determine the outer contours of all shapes in each target video based on the multiple target video frames and the color threshold range of the detection target; wherein a first detection target among the detection targets is set at the master end of the master-slave system, and a second detection target is set at the slave end of the master-slave system; a target contour determination module, configured to traverse the outer contour of each shape to determine the contour of the detection target and the center coordinates of the detection target in each target video frame; an offset determination module, configured to determine, for each target video frame, the two-dimensional coordinate offset of the detection target in the frame based on the center coordinates of the detection target in the frame and the initial center coordinates of the detection target in the first target video frame; and a delay determination module, configured to determine the master-slave operation delay of the master-slave system at different stages based on the two-dimensional coordinate offset of the detection target in each target video frame. By setting the two-dimensional coordinate offset of the detection target at the master and slave ends during the movement, the delay between the master and slave can be calculated quickly and accurately, thereby achieving visual measurement and evaluation of the delay.

[0119] See also Figure 4 , Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 4 As shown in FIG, the electronic device 400 includes a processor 410 , a memory 420 and a bus 430 .

[0120] The memory 420 stores machine-readable instructions executable by the processor 410. When the electronic device 400 is running, the processor 410 communicates with the memory 420 via the bus 430. When the machine-readable instructions are executed by the processor 410, the above-mentioned Figure 1 as well as Figure 2 The steps of the method for testing the operation delay of the master-slave system in the method embodiment shown are specifically implemented in accordance with the method embodiment and will not be described in detail here.

[0121] The embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the computer program can execute the above-mentioned Figure 1 as well as Figure 2 The steps of the method for testing the operation delay of the master-slave system in the method embodiment shown are specifically implemented in accordance with the method embodiment and will not be described in detail here.

[0122] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0123] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. There may be other division methods in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed may be through some communication interface, indirect coupling or communication connection of devices or units, which may be electrical, mechanical or other forms.

[0124] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0125] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0126] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0127] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The scope of protection of the present application is not limited thereto. Although the present application has been described in detail with reference to the above-mentioned embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above-mentioned embodiments within the technical scope disclosed in the present application, or perform equivalent replacements for some of the technical features thereof. These modifications, changes, or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A method for testing the operation delay of a master-slave system, characterized in that: The operation delay testing method includes: Performing color space conversion on multiple video frames during operation of the master-slave system to determine a target video frame, and determining the outer contours of all shapes in each target video based on the multiple target video frames and a color threshold range of a detection target; wherein a first detection target among the detection targets is set at a master end of the master-slave system, and a second detection target is set at a slave end of the master-slave system; Traversing the outer contour of each shape to determine the contour of the detection target and the center coordinates of the detection target in each target video frame; For each target video frame, determining a two-dimensional coordinate offset of the detection target in the frame based on the center coordinates of the detection target in the frame and the initial center coordinates of the detection target in the first target video frame; Based on the two-dimensional coordinate offset of the detection target in each target video frame, the master-slave operation delay of the master-slave system in different stages is determined.

2. The operation delay testing method according to claim 1, wherein: Determining the outer contours of all shapes in each target video based on the plurality of target video frames and the color threshold range of the detection target includes: Extracting color regions from the plurality of target video frames based on the color threshold range, and generating a mask image based on the color regions; Performing binarization processing, noise removal processing, and hole filling processing on the mask image of each target video frame to determine a binary image of each target video frame; The outer contours of all shapes are extracted from the binary image of each target video frame.

3. The operation delay testing method according to claim 1, wherein: The traversing the outer contour of each shape to determine the contour of the detection target and the center coordinates of the detection target in each target video frame includes: Detecting whether the area of ​​the region corresponding to the outer contour of the shape is within the area range of the region corresponding to the detection target; If not, skip the outer contour of this shape and execute the outer contour of the next shape; If so, detecting whether the equivalent radius of the outer contour of the shape is within the equivalent radius range corresponding to the detection target; If it is within the equivalent radius, the outer contour of the shape is taken as a candidate outer contour; if it is not within the equivalent radius, the outer contour of the shape is skipped and the outer contour of the next shape is executed; Check whether the axis ratio and the number of corner points of the candidate outer contour meet the requirements of the detection target. If they do, determine the candidate outer contour as the candidate outer contour and determine the center coordinates.

4. The operation delay testing method according to claim 1, wherein: For each target video frame, determining a two-dimensional coordinate offset of the detection target in the frame based on the center coordinates of the detection target in the frame and the initial center coordinates of the detection target in the first target video frame includes: Determining a two-dimensional coordinate offset of the first detection target in the frame based on the center coordinates of the first detection target in the frame and the initial center coordinates of the first detection target; Based on the center coordinates of the second detection target in the frame and the initial center coordinates of the second detection target, a two-dimensional coordinate offset of the second detection target in the frame is determined.

5. The operation delay testing method according to claim 1, wherein: The determining of the master-slave operation delays of the master-slave system at different stages based on the two-dimensional coordinate offset of the detection target in each target video frame includes: Constructing a three-dimensional trajectory graph based on the changes in the two-dimensional coordinate offset of the first detection target and the offset of the second detection target in each target video frame over time; Based on the time points at which the two-dimensional coordinate offset of the first detection target changes significantly and the time points at which the two-dimensional coordinate offset of the second detection target changes significantly in the three-dimensional trajectory diagram, the master-slave operation delays of the master-slave system at different stages are determined.

6. The operation delay testing method according to claim 5, characterized in that: Determining the master-slave operation delays of the master-slave system at different stages based on a time point at which the two-dimensional coordinate offset of the first detection target in the three-dimensional trajectory graph significantly changes and a time point at which the two-dimensional coordinate offset of the second detection target significantly changes, including: Determining a first master-slave operation delay of the master-slave system based on a time point at which the two-dimensional coordinate offset of the first detection target in the three-dimensional trajectory graph first significantly changes and a time point at which the two-dimensional coordinate offset of the second detection target first significantly changes; The master-slave operation delays of the master-slave system at different stages are further determined based on the time point of the next significant change of the first detection target and the time point of the next significant change of the second detection target.

7. The operation delay testing method according to claim 1, wherein: The detection targets are circular marks and square marks, and the shapes of the first detection target and the second detection target are consistent or inconsistent.

8. An operation delay test device for a master-slave system, characterized in that: The operation delay testing device comprises: a video frame processing module, configured to perform color space conversion on multiple video frames during operation of the master-slave system to determine a target video frame, and determine the outer contours of all shapes in each target video based on the multiple target video frames and the color threshold range of the detection target; wherein a first detection target among the detection targets is set at the master end of the master-slave system, and a second detection target is set at the slave end of the master-slave system; A target contour determination module is used to traverse the outer contour of each shape to determine the contour of the detection target and the center coordinates of the detection target in each target video frame; an offset determination module, configured to determine, for each target video frame, a two-dimensional coordinate offset of the detection target in the frame based on the center coordinates of the detection target in the frame and the initial center coordinates of the detection target in the first target video frame; The time delay determination module is used to determine the master-slave operation time delay of the master-slave system at different stages based on the two-dimensional coordinate offset of the detection target in each target video frame.

9. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the memory communicate through the bus. When the processor is running, the machine-readable instructions execute the steps of the operation delay testing method of the master-slave system as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for testing the operation delay of the master-slave system according to any one of claims 1 to 7 are executed.

Images