Calibration Method for Image Display Delay and Related Products

Abstract

The present application discloses a calibration method for image display latency and related products. The method is used to determine the display latency of a display device for displaying an image captured by an image capture device. The method includes: obtaining n first images, where n is an integer greater than 1, and the n first images include n images captured by the image capture device for a target object in a case where there is relative motion between the image capture device and the target object; obtaining a target display result, where the target display result includes a display result obtained by displaying the n first images on the display device in the acquisition order of the n first images; obtaining a first relationship based on the position of the target object in the n first images; obtaining a second relationship based on the position of the target object in the target display result; and obtaining the display latency of the display device based on the time difference between the first relationship and the second relationship.

Description

Technical Field

[0001] This application relates to the technical field of medical image processing, and in particular, to a method for calibrating image display latency and related products. Background Art

[0002] There is a communication connection between an ultrasound acquisition device and a display device. After the ultrasound acquisition device acquires an image, the ultrasound acquisition device transmits the acquired ultrasound image to the display device through this communication connection for the display device to display the ultrasound image. In this process, there is a time difference between when the ultrasound acquisition device acquires the ultrasound image and when the display device displays the ultrasound image, and this time difference is the display latency. Therefore, how to determine the display latency is of great significance. Summary of the Invention

[0003] This application provides a method for calibrating image display latency and related products to determine the display latency of the image acquired by an image acquisition device and displayed by a display device. Among them, the related products include a calibration device for image display latency, an electronic device, a computer-readable storage medium, and a computer program product.

[0004] In a first aspect, a method for calibrating image display latency is provided. The method is used to determine the display latency of the image acquired by an image acquisition device and displayed by a display device, and the method includes:

[0005] Obtain n first images, where n is an integer greater than 1, and the n first images include n images acquired by the image acquisition device for the target object when there is relative movement between the image acquisition device and the target object;

[0006] Obtain a target display result, where the target display result includes the display result obtained by displaying the n first images on the display device in the acquisition order of the n first images;

[0007] Based on the position of the target object in the n first images, obtain a first relationship, where the first relationship represents the relationship between the position of the target object in the image acquired by the image acquisition device and time;

[0008] Based on the position of the target object in the target display result, obtain a second relationship, where the second relationship represents the relationship between the position of the target object in the image displayed by the display device and time;

[0009] Based on the time difference between the first relationship and the second relationship, obtain the display latency of the display device.

[0010] Combined with any embodiment of the present application, obtaining the display delay of the display device based on the time difference between the first relationship and the second relationship includes:

[0011] Determine a first position corresponding to a first time from the first relationship;

[0012] Determine at least one candidate time corresponding to the first position from the second relationship;

[0013] Determine the time with the smallest difference from the first time among the at least one candidate time as the second time;

[0014] Obtain the display delay according to the time difference between the first time and the second time.

[0015] Combined with any embodiment of the present application, before determining the first position corresponding to the first time from the first relationship, the method further includes:

[0016] Based on the first relationship, determine a first curve that matches the first relationship;

[0017] Determine a point on the first curve with a slope greater than or equal to a slope threshold as a reference point;

[0018] Determine the time corresponding to the reference point in the first relationship as the first time.

[0019] Combined with any embodiment of the present application, the n first images include n images collected by the image acquisition device when the target object is moving;

[0020] Before obtaining the display delay of the display device based on the time difference between the first relationship and the second relationship, the method further includes:

[0021] Obtain a third relationship, where the third relationship is the relationship between the position of the target object in the world coordinate system and time;

[0022] Using a first conversion relationship, convert the positions in the display results of the n first images of the target object to the world coordinate system to obtain n converted positions, where the first conversion relationship is used to convert the positions in the display results of the display device to the positions in the world coordinate system;

[0023] Based on the n converted positions and the display times of the display results of the n first images, obtain a fourth relationship, where the fourth relationship is the relationship between the position of the target object in the world coordinate system and time;

[0024] Obtaining the display latency of the display device based on the time difference between the first relationship and the second relationship includes:

[0025] Determining a first latency between the first relationship and the third relationship;

[0026] Determining a second latency between the fourth relationship and the third relationship;

[0027] Obtaining the display latency based on the difference between the second latency and the first latency.

[0028] Combined with any embodiment of the present application, obtaining the display latency based on the time difference between the first latency and the second latency includes:

[0029] Determining the time difference between the first latency and the second latency as the display latency.

[0030] Combined with any embodiment of the present application,

[0031] Determining the first latency between the first relationship and the third relationship includes:

[0032] Based on the first relationship, determining a first curve that matches the first relationship;

[0033] Based on the third relationship, determining a third curve that matches the third relationship;

[0034] Intercepting a curve with a preset time span from the first curve as a fourth curve;

[0035] Determining a fifth curve that matches the fourth curve from the third curve;

[0036] Obtaining the first latency based on the latency between the fourth curve and the fifth curve.

[0037] In a second aspect, a calibration device for image display latency is provided. The calibration device for image display latency is used to determine the display latency of an image collected by an image acquisition device and displayed by a display device. The calibration device for image display latency includes:

[0038] An acquisition unit configured to acquire n first images, where n is an integer greater than 1, and the n first images include n images acquired by the image acquisition device for the target object when there is relative motion between the image acquisition device and the target object;

[0039] The acquisition unit is configured to acquire a target display result, where the target display result includes a display result obtained by displaying the n first images on the display device in the acquisition order of the n first images;

[0040] A processing unit, configured to obtain a first relationship based on the position of the target object in the n first images, where the first relationship characterizes the relationship between the position of the target object in the images captured by the image acquisition device and time;

[0041] The processing unit is configured to obtain a second relationship based on the position of the target object in the target display result, where the second relationship characterizes the relationship between the position of the target object in the image displayed by the display device and time;

[0042] The processing unit is configured to obtain the display delay of the display device based on the time difference between the first relationship and the second relationship.

[0043] Combined with any embodiment of the present application, the processing unit is specifically configured to:

[0044] Determine a first position corresponding to a first time from the first relationship;

[0045] Determine at least one candidate time corresponding to the first position from the second relationship;

[0046] Determine the time with the smallest difference from the first time among the at least one candidate time as the second time;

[0047] Obtain the display delay according to the time difference between the first time and the second time.

[0048] Combined with any embodiment of the present application, the processing unit is further configured to:

[0049] Based on the first relationship, determine a first curve that matches the first relationship;

[0050] Determine a point with a slope greater than or equal to a slope threshold as a reference point from the first curve;

[0051] Determine the time corresponding to the reference point in the first relationship as the first time.

[0052] Combined with any embodiment of the present application, the n first images include n images captured by the image acquisition device when the target object is moving;

[0053] The processing unit is further configured to:

[0054] Obtain a third relationship, where the third relationship is the relationship between the position of the target object in the world coordinate system and time;

[0055] Using the first conversion relationship, convert the position of the target object in the display results obtained from the n first images to the world coordinate system, obtaining n converted positions, where the first conversion relationship is used to convert the position in the display results of the display device to the position in the world coordinate system;

[0056] Based on the n converted positions and the display times of the display results of the n first images, obtain a fourth relationship, where the fourth relationship is the relationship between the position of the target object in the world coordinate system and time;

[0057] The processing unit is specifically configured to:

[0058] Based on the first relationship, determine a first curve that matches the first relationship;

[0059] Based on the third relationship, determine a third curve that matches the third relationship;

[0060] Intercept a curve with a preset time span from the first curve as the fourth curve;

[0061] Determine a fifth curve that matches the fourth curve from the third curve;

[0062] Based on the time delay between the fourth curve and the fifth curve, obtain the first time delay.

[0063] Combined with any embodiment of the present application, the processing unit is specifically configured to:

[0064] Determine the time difference between the first time delay and the second time delay as the display time delay.

[0065] Combined with any embodiment of the present application, the processing unit is specifically configured to:

[0066] Based on the second relationship, determine a second curve, where the second curve is a curve that matches the second relationship;

[0067] Based on the third relationship, determine a third curve, where the third curve is a curve that matches the third relationship;

[0068] Intercept a curve with a preset time span from the second curve as the fourth curve;

[0069] Determine a fifth curve that matches the fourth curve from the third curve;

[0070] Based on the time delay between the fourth curve and the fifth curve, obtain the second time delay.

[0071] In a third aspect, an electronic device is provided, including: a processor and a memory, where the memory is configured to store computer program code, and the computer program code includes computer instructions. When the processor executes the computer instructions, the electronic device executes the method according to the first aspect and any one of its possible implementation manners as described above.

[0072] In a fourth aspect, another electronic device is provided, including: a processor, a sending device, an input device, an output device, and a memory, where the memory is configured to store computer program code, and the computer program code includes computer instructions. When the processor executes the computer instructions, the electronic device executes the method according to the first aspect and any one of its possible implementation manners as described above.

[0073] In a fifth aspect, a computer-readable storage medium is provided, in which a computer program is stored, and the computer program includes program instructions. When the program instructions are executed by a processor, the processor is caused to execute the method according to the first aspect and any one of its possible implementation manners as described above.

[0074] In a sixth aspect, a computer program product is provided, and the computer program product includes a computer program or instructions. When the computer program or instructions run on a computer, the computer is caused to execute the method according to the first aspect and any one of its possible implementation manners as described above.

[0075] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and do not limit this application.

[0076] In this application, the n first images include n images collected by an image acquisition device for a target object in a case where there is relative movement between the image acquisition device and the target object. After the calibration device acquires the n first images, based on the positions of the target object in the n first images, a first relationship is obtained, where the first relationship represents the relationship between the position of the target object in the images collected by the image acquisition device and time. The target display result includes a display result obtained by displaying the n first images on a display device in the acquisition order of the n first images. After the calibration device acquires the target result, based on the position of the target object in the target display result, a second relationship is obtained, where the second relationship represents the relationship between the position of the target object in the images displayed by the display device and time. Finally, based on the time difference between the first relationship and the second relationship, the display delay of the display device can be obtained. Description of the Drawings

[0077] In order to more clearly illustrate the technical solutions in the embodiments of this application or the background art, the following will describe the drawings required to be used in the embodiments of this application or the background art.

[0078] The accompanying drawings here are incorporated into the specification and form a part of this specification. These drawings illustrate embodiments consistent with the present application and, together with the specification, are used to explain the technical solutions of the present application.

[0079] Figure 1 It is a schematic flowchart of a method for calibrating the display delay of an ultrasonic image provided by an embodiment of the present application;

[0080] Figure 2 It is a schematic diagram of a first relationship and a second relationship before calibration provided by an embodiment of the present application;

[0081] Figure 3 It is a schematic diagram of a first relationship and a second relationship after calibration provided by an embodiment of the present application;

[0082] Figure 4 It is a schematic diagram of a scenario where an ultrasonic probe is used to scan a calibration tooling provided by an embodiment of the present application;

[0083] Figure 5 It is a schematic structural diagram of a device for calibrating the display delay of an image provided by an embodiment of the present application;

[0084] Figure 6 It is a schematic hardware structure diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0085] In order to enable those skilled in the art to better understand the solutions of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the protection scope of the present application.

[0086] The terms "first", "second", etc. in the specification, claims and above-mentioned accompanying drawings of the present application are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally further includes steps or units not listed, or optionally further includes other steps or units inherent to these processes, methods, products or devices.

[0087] References to "embodiments" in this specification mean that a particular feature, structure, or characteristic described in connection with the embodiments can be included in at least one embodiment of the present application. The phrase appears in various places in the specification and does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art will explicitly and implicitly understand that the embodiments described herein can be combined with other embodiments. It should be understood that in the present application, "at least one (item)" means one or more, "a plurality" means two or more, and "at least two (items)" means two, three, or more.

[0088] The execution subject of the embodiments of the present application is a calibration device for the display delay of ultrasonic images (hereinafter simply referred to as the calibration device). Among them, the calibration device can be any electronic device that can execute the technical solutions disclosed in the method embodiments of the present application. By performing the calibration of the display delay of ultrasonic images, the calibration device can determine the display delay for the display device to display the ultrasonic images collected by the ultrasonic acquisition device. Specifically, there is a communication connection between the ultrasonic acquisition device and the display device. After the ultrasonic acquisition device acquires an image, the ultrasonic acquisition device transmits the acquired ultrasonic image to the display device through this communication connection for the display device to display the ultrasonic image. During this process, there is a time difference between the ultrasonic acquisition device acquiring the ultrasonic image and the display device displaying the ultrasonic image, and this time difference is the display delay. For example, the ultrasonic acquisition device acquires an ultrasonic image at time t1, and the display device displays the ultrasonic image at time t2. At this time, the display delay is t2 - t1. Optionally, the calibration device can be one of the following: a mobile phone, a computer, a tablet computer, a wearable intelligent device.

[0089] It should be understood that the method embodiments of the present application can also be implemented by a processor executing computer program code. The embodiments of the present application will be described below with reference to the accompanying drawings in the embodiments of the present application. Please refer to Figure 1 , Figure 1 which is a schematic flowchart of a method for calibrating the display delay of ultrasonic images provided by the embodiments of the present application.

[0090] 101. Obtain n first ultrasonic images.

[0091] In an embodiment of the present application, n is an integer greater than 1. The n first images include n images of the target object collected by the image acquisition device when there is relative motion between the image acquisition device and the target object. The relative motion between the image acquisition device and the target object may be that the target object is in a stationary state and the image acquisition device is in a moving state. The relative motion between the image acquisition device and the target object may also be that the target object is in a moving state and the image acquisition device is in a stationary state. The relative motion between the image acquisition device and the target object may further be that both the target object and the image acquisition device are in a moving state. The n first images are all collected by the image acquisition device, and the acquisition times of any two first images are different.

[0092] In an embodiment of the present application, the target object may be any object. For example, the target object is a line, or for another example, the target object is a bump. The imaging device may be any device capable of imaging. Optionally, the imaging device is an ultrasonic probe, and in this case, the first image is an ultrasonic image.

[0093] In one implementation manner of obtaining the n first images, the calibration device receives the n first images input by the user through the input component. Among them, the input component includes: a mouse, a keyboard, a touch screen, a touchpad, and an audio input device.

[0094] In another implementation manner of obtaining the n first images, the calibration device receives the n first images sent by the terminal. Among them, the terminal includes: a mobile phone, a computer, a tablet computer, and a smart wearable device.

[0095] 102. Obtain the target display result.

[0096] In an embodiment of the present application, the target display result includes the display result obtained by displaying the n first images on the display device in the acquisition order of the n first images. Specifically, the display device displays the n first images in the acquisition order of the n first images, and the display result of the n first images can be obtained, which is the target display result. For example, the n first images include image a and image b. Among them, the acquisition time of image a is earlier than that of image b. Then the acquisition order of the n first images is to acquire image a first and then image b. When the display device displays the n first images, it first displays image a and then image b. At this time, the target display result includes the display result of image a and the display result of image b. Optionally, the display result of the display device for displaying the first image is to display the image, and the target display result includes n display images corresponding to the n first images. For example, the n first images include image a and image b. Among them, the display image corresponding to image a is display image c, and the display image corresponding to image b is display image d. At this time, the target display result includes display image c and display image d.

[0097] Optionally, there is a communication connection between the image acquisition device and the display device. After the image acquisition device acquires the first image, it sends the first image to the display device through this communication connection so that the display device can display the first image. Optionally, the display device is a device including a display device. For example, the display device is a computer including a display, and the display is the display device.

[0098] In an implementation manner of obtaining the target display result, the calibration device receives the target display result input by the user through the input component.

[0099] In another implementation manner of obtaining the target display result, the calibration device receives the target display result sent by the terminal.

[0100] 103. Based on the position of the above-mentioned target object in the above-mentioned n first images, obtain the first relationship.

[0101] In the embodiment of the present application, the first relationship characterizes the relationship between the position of the target object in the image acquired by the image acquisition device and time, that is, based on the first relationship, the position of the target object in the image acquired by the image acquisition device at any time can be determined. In a possible implementation manner, the calibration device determines the positions of the target object in the n first images to obtain n imaging positions. Based on the n imaging positions and the acquisition times corresponding to the n first images, n imaging data points are obtained, where the imaging data points include the imaging positions and the acquisition times corresponding to the imaging positions. Fit the n imaging data points to obtain the first relationship.

[0102] 104. Based on the position of the above-mentioned target object in the above-mentioned target display result, obtain the second relationship.

[0103] In the embodiment of the present application, the second relationship characterizes the relationship between the position of the target object in the image displayed by the display device and time, that is, based on the second relationship, the position of the target object in the image displayed by the display device at any time can be determined. In a possible implementation manner, the calibration device determines the positions of the target object in the target display result to obtain n display positions. Based on the n display positions and the display times corresponding to the n first images, n display data points are obtained, where the display data points include the display positions and the display times corresponding to the display positions. Fit the n display data points to obtain the second relationship.

[0104] 105. Based on the time difference between the above-mentioned first relationship and the above-mentioned second relationship, obtain the above-mentioned display delay of the above-mentioned display device.

[0105] Considering that there is a time delay between the time when the image acquisition device acquires an image and the time when the display device displays the image, the first relationship and the second relationship should be different. Specifically, there is a time difference between the first relationship and the second relationship. Therefore, based on the time difference between the first relationship and the second relationship, the calibration device can obtain the display time delay of the display device.

[0106] In a possible implementation manner, the calibration device uses the time difference between the first relationship and the second relationship as the display time delay of the display device. In another possible implementation manner, the calibration device determines the time difference between the first relationship and the second relationship, and determines the sum of the time difference and a first preset value to obtain the display time delay, where the first preset value is a positive number. In still another possible implementation manner, the calibration device determines the time difference between the first relationship and the second relationship, and determines the product of the time difference and a second preset value to obtain the display time delay, where the second preset value is a positive number.

[0107] In the embodiments of the present application, the n first images include n images acquired by the image acquisition device for the target object when there is relative movement between the image acquisition device and the target object. After the calibration device acquires the n first images, based on the position of the target object in the n first images, a first relationship is obtained, where the first relationship represents the relationship between the position of the target object in the image acquired by the image acquisition device and time. The target display result includes the display result obtained by displaying the n first images on the display device in the acquisition order of the n first images. After the calibration device acquires the target result, based on the position of the target object in the target display result, a second relationship is obtained, where the second relationship represents the relationship between the position of the target object in the image displayed by the display device and time. Finally, based on the time difference between the first relationship and the second relationship, the display time delay of the display device can be obtained.

[0108] As an alternative implementation manner, during the execution of step 105, the calibration device performs the following steps:

[0109] 201. Determine a first position corresponding to a first time from the above first relationship.

[0110] In the embodiments of the present application, the first time is any time in the first relationship. In the first relationship, the position and time are in one-to-one correspondence, and the position corresponding to the first time is the first position.

[0111] 202. Determine at least one candidate time corresponding to the above first position from the above second relationship.

[0112] Since in the second relationship, there is at least one different time corresponding to the same position. For example, in the second relationship, the positions corresponding to time t1 and time t2 are both p, that is, the position of the target object in the image displayed on the display device at time t1 is the same as the position of the target object in the image displayed on the display device at time t2. Therefore, the calibration device can determine at least one time corresponding to the first position from the second relationship, which is at least one candidate time.

[0113] 203. Determine the time with the smallest difference from the above first time among the above at least one candidate time as the second time.

[0114] Considering that the position of the target object in the first image is usually the same as the position of the target object in the displayed image of the first image, and there is a time corresponding to the position of the target object in the first image in the first relationship, and there is a time corresponding to the position of the target object in the displayed image in the second relationship. For the sake of convenience of expression, hereinafter, the time corresponding to the position of the target object in the first image in the first relationship and the time corresponding to the position of the target object in the displayed image in the second relationship are referred to as times with a corresponding relationship. For example, the position of the target object in the first image a is the first position, where the acquisition time of the first image a is t1. In the first relationship, the time corresponding to the first position is t1. The display device displays the first image a at time t2 to obtain a displayed image b. At this time, the position of the target object in the displayed image b is also the first position, and the time corresponding to the first position in the second relationship is t2. At this time, t1 and t2 are times with a corresponding relationship.

[0115] Since time corresponds to position in the first relationship and the second relationship, the calibration device can determine the times with a corresponding relationship from the first relationship and the second relationship with the position as a bridge. Thus, after the calibration device determines the first position corresponding to the first time from the first relationship, it can further use the first position as a bridge to determine the time with a corresponding relationship to the first time from the second relationship. Therefore, after the calibration device determines at least one candidate time from the second relationship, it can further determine the time with a corresponding relationship to the first time from the at least one candidate time.

[0116] When the number of candidate times is 1, the calibration device determines that the candidate time is the time with a corresponding relationship to the first time, which is the second time. When the number of candidate times is 2, it is necessary to determine a time with a corresponding relationship to the first time from at least two candidate times. Since the display delay of the display device is usually small and the time difference between the two times with a corresponding relationship in the first relationship and the second relationship is small, the calibration device determines the time with the smallest difference from the first time among the at least one candidate time as the time with a corresponding relationship to the first time, which is the second time.

[0117] 204. Obtain the above display delay according to the time difference between the above first time and the above second time.

[0118] In a possible implementation manner, the calibration device determines the time difference between the first time and the second time to obtain the display delay. In another possible implementation manner, the calibration device determines the time difference between the first time and the second time, determines the sum of the time difference and a first preset value to obtain the display delay. In still another possible implementation manner, the calibration device determines the time difference between the first time and the second time, determines the product of the time difference and a first preset value to obtain the display delay.

[0119] As an optional implementation manner, before the calibration device executes step 201, it also executes the following steps:

[0120] 301. Based on the above first relationship, determine a first curve that matches the above first relationship.

[0121] 302. Determine a point on the above first curve whose slope is greater than or equal to a slope threshold as a reference point.

[0122] 303. Determine the time corresponding to the above reference point in the above first relationship as the above first time.

[0123] A large slope of a point on the first curve indicates a large change in the position corresponding to the point. Therefore, before step 201, the point is used as a reference point, and the time corresponding to the point is used as the first time. Then, through steps 201 to 204, determine the first position corresponding to the first time, and obtain the display delay by determining the time difference between the time when the imaging device acquires the first image corresponding to the first position and the time when the display device displays the first image corresponding to the first position. It is possible to obtain the display delay by determining the time difference between the time when the imaging device acquires the first image corresponding to a position with a large change and the time when the display device displays the first image corresponding to a position with a large change. Thereby, the accuracy of the display delay can be improved.

[0124] As an optional implementation manner, the n first images include n images acquired by an image acquisition device when the target object is moving. Optionally, the image acquisition device is in a stationary state during the process of acquiring the n first images. In this implementation manner, before the calibration device executes step 105, it also executes the following steps:

[0125] 401. Obtain a third relationship.

[0126] In the embodiments of the present application, the third relationship is the relationship between the position of the target object in the world coordinate system and time. That is, based on the third relationship, the position of the target object in the world coordinate system at any time can be determined. In a possible implementation manner, the calibration device determines m moving positions of the target object during the movement. Based on the m moving positions and the moving times corresponding to the m moving positions, m moving data points are obtained, where the moving data points include the moving positions and the moving times corresponding to the moving positions. The n moving data points are fitted to obtain the third relationship.

[0127] 402. Using the first conversion relationship, convert the position of the target object in the display results of the above n first images to the above world coordinate system to obtain n converted positions.

[0128] In the embodiments of the present application, the first conversion relationship is used to convert the position in the display result of the display device to the position in the world coordinate system. Therefore, the calibration device can use the first conversion relationship to convert the position of the target object in the display results obtained from the n first images to the world coordinate system to obtain n converted positions.

[0129] 403. Based on the above n converted positions and the display times of the display results obtained from the above n first images, obtain a fourth relationship.

[0130] In the embodiments of the present application, the fourth relationship is the relationship between the position of the target object in the world coordinate system and time, that is, based on the fourth relationship, the position of the target object in the world coordinate system at any time can be determined. It should be understood that since the n converted positions are obtained based on the positions of the target object in the display results of the n first images, and there is a time delay between the position of the target object in the display result of the first image and the position of the target object in the world coordinate system, there is a time delay between the third relationship and the fourth relationship. In a possible implementation manner, the calibration device obtains n conversion data points based on the n converted positions and the display times of the n first images, where the conversion data points include the converted positions and the display times corresponding to the converted positions. The n conversion data points are fitted to obtain the fourth relationship.

[0131] After obtaining the fourth relationship, the calibration device performs the following steps during the execution of step 105:

[0132] 404. Determine the first time delay between the above first relationship and the above third relationship.

[0133] In a possible implementation manner, the calibration device determines the time difference between the first relationship and the third relationship to obtain the first time delay.

[0134] 405. Determine the second time delay between the above fourth relationship and the above third relationship.

[0135] In a possible implementation, the calibration device determines the time difference between the third relationship and the fourth relationship to obtain a second time delay.

[0136] 406. Based on the difference between the second time delay and the first time delay, the display time delay is obtained.

[0137] In a possible implementation, the calibration device determines the difference between the first time delay and the second time delay to obtain the display time delay. In another possible implementation, the calibration device determines the time difference between the first time delay and the second time delay, determines the sum of the time difference and a first preset value to obtain the display time delay. In yet another possible implementation, the calibration device determines the time difference between the first time delay and the second time delay, determines the product of the time difference and a second preset value to obtain the display time delay.

[0138] Considering the display error of the display device, it may cause the position of the target object in the display result of the first image to be different from the position of the target object in the first image. For example, due to the difference between the display resolution of the display device and the resolution of the first image, the position of the target object in the display result of the first image is different from the position of the target object in the first image. When this situation occurs, it will cause the positions corresponding to the same first image in the first relationship and the second relationship to be different. For example, the display result obtained by the display device displaying the first image a is the display image b. Then in the first relationship, the position corresponding to the first image a is the position of the target object in the first image a. In the second relationship, the position corresponding to the display image b is the position of the target object in the display image b. Since the display image b is the image corresponding to the first image a, the position corresponding to the first image a in the second relationship is the position corresponding to the display image b, that is, in the second relationship, the position corresponding to the first image a is the position of the target object in the display image b. If the position of the target object in the first image a is position c and the position of the target object in the display image b is position d, then in the first relationship, the position corresponding to the first image a is position c, and in the second relationship, the position corresponding to the first image a is position d. If the display device has a display error, then position c and position d are different. If the positions corresponding to the same first image in the first relationship and the second relationship are different, it will further cause a large error in the display time delay determined based on the time difference between the first relationship and the second relationship.

[0139] Therefore, in this embodiment, the calibration device first uses the first conversion relationship to convert the position of the target object in the display results obtained from n first images into the world coordinate system, obtaining n converted positions, thereby removing the display error of the display device. Then, based on the n converted positions and the display times of the display results of the n first images, a fourth relationship is obtained. Next, the first time delay between the first relationship and the third relationship is determined, where the first time delay represents the time delay of the image acquisition device for acquiring the position of the target object in the world coordinate system. For example, at time t1, the position of the target object in the world coordinate system is p1, and the image acquisition device acquires p1 at time t2. At this time, the first time delay is t2 - t1. The second time delay between the fourth relationship and the third relationship is determined, where the second time delay can represent the time delay of the display device for displaying the position of the target object in the world coordinate system. For example, at time t1, the position of the target object in the world coordinate system is p1, and the display device displays p1 at time t3. At this time, the second time delay is t3 - t1. Since the second time delay = the first time delay + the display time delay, the calibration device obtains the display time delay based on the time difference between the second time delay and the first time delay. This can reduce the influence of the display error of the display device on determining the display time delay, thereby improving the accuracy of the display time delay.

[0140] As an alternative embodiment, during the execution of step 405, the calibration device performs the following steps:

[0141] 501. Based on the above first relationship, determine a first curve that matches the above first relationship.

[0142] In the embodiments of the present application, the first curve matches the first relationship, that is, the first relationship can be represented by the first curve.

[0143] 502. Based on the above third relationship, determine a third curve that matches the above third relationship.

[0144] In the embodiments of the present application, the third curve matches the third relationship, that is, the third relationship can be represented by the first curve.

[0145] 503. Intercept a curve with a preset time span from the above first curve as the fourth curve.

[0146] 504. Determine a fifth curve that matches the above fourth curve from the above third curve.

[0147] 505. Based on the time delay between the above fourth curve and the above fifth curve, obtain the above first time delay.

[0148] Optionally, the matching of the fourth curve and the fifth curve may be that the coincidence degree of the fourth curve and the fifth curve is greater than or equal to the coincidence threshold. Optionally, the matching of the fourth curve and the fifth curve may also be that the difference between the positions of the target corresponding to the inflection point of the fourth curve and the positions of the target corresponding to the inflection point of the fifth curve is less than or equal to the inflection point threshold. For example, the fourth curve includes an inflection point a, and the fifth curve includes an inflection point b, where the inflection point a represents that the position of the target in the world coordinate system at time t1 is p1, and the inflection point b represents that the position of the target in the world coordinate system at time t2 is p2. Then the position of the target corresponding to the inflection point of the fourth curve is p1, and the position of the target corresponding to the inflection point of the fifth curve is p2.

[0149] In this implementation, the data volume of the fourth curve is smaller than that of the first curve, and the data volume of the fifth curve is smaller than that of the third curve. Therefore, after the calibration device obtains the fourth curve by executing step 503 and obtains the fifth curve by executing step 504, based on the time delay between the fourth curve and the fifth curve, the first time delay is obtained, which can reduce the data processing volume for determining the first time delay between the first curve and the third curve and improve the processing speed.

[0150] On the one hand, considering that the larger the time span of the fourth curve, the larger the data volume of the fourth curve, and the larger the data volume of the fifth curve. Correspondingly, determining the first time delay based on the time delay between the fourth curve and the fifth curve can improve the accuracy of the first time delay. On the other hand, the larger the time span of the fourth curve, the larger the data volume of the fourth curve, and the larger the data volume of the fifth curve. Correspondingly, based on the time delay between the fourth curve and the fifth curve, the data processing volume for obtaining the first time delay is larger. Therefore, the specific value of the preset value can be set according to actual needs, and the present application does not limit this.

[0151] Optionally, after determining the display time delay based on the method described above, the display time of the display device can be calibrated based on the display time delay, thereby reducing the display error caused by the display time delay. For example, when the display device displays the ultrasonic image collected by the ultrasonic acquisition device, the difference between the current time and the display time delay is determined as the acquisition time of the ultrasonic image, and the acquisition time is displayed while the ultrasonic image is displayed.

[0152] Optionally, after calibrating the display time of the display device based on the display time delay, the time delay between the first relationship and the second relationship can be reduced. Please refer to Figure 2 , Figure 2 which is a schematic diagram of a first relationship and a second relationship before calibration provided by an embodiment of the present application. In Figure 2 the shown coordinate system, the horizontal axis represents time, and the vertical axis represents the position of the target. In Figure 2 , the solid line represents the first relationship, and the dashed line represents the second relationship. As Figure 2As shown, there is a time delay between the first relationship and the second relationship before calibration, which is the above-mentioned display time delay. Please refer to Figure 3 , Figure 3 which is a schematic diagram of the first relationship and the second relationship after calibration provided by an embodiment of the present application. In the Figure 3 shown coordinate system, the horizontal axis represents time and the vertical axis represents the position of the target object. In Figure 3 , the solid line represents the first relationship and the dashed line represents the second relationship. As Figure 3 shown, after calibration, the time delay between the first relationship and the second relationship is reduced compared to the time delay before calibration.

[0153] Optionally, the image acquisition device is an ultrasonic probe, and the target object belongs to the calibration tooling, that is, the target object is a structure on the calibration tooling. The ultrasonic probe can obtain n first images by scanning the calibration tooling.

[0154] Optionally, the design requirements of the calibration tooling are as follows:

[0155] 1) The ultrasonic probe requires a medium during use, and water is the most convenient medium to obtain, so the calibration tooling can be filled with water;

[0156] 2) The scanning area of the ultrasonic probe has a certain depth and width, so the calibration tooling has a height matching the depth and a length matching the width;

[0157] 3) To facilitate the ultrasonic probe to move back and forth for scanning, the area of the region on the calibration tooling scanned by the ultrasonic probe should be greater than or equal to the area threshold;

[0158] 4) Since the resolution of the ultrasonic images obtained by scanning with the ultrasonic probe is low and there are many artifacts, a thin line can be selected as the target object in the calibration tooling. This can reduce the artifacts in the ultrasonic images, and thus improve the accuracy of determining the position of the target object in the n first images. Optionally, the diameter of the thin line is less than 1 mm. Optionally, the thin line is not easily broken;

[0159] Optionally, Figure 4 which is a schematic diagram of a scenario for using an ultrasonic probe to scan a calibration tooling provided by an embodiment of the present application. As Figure 4 shown, the user can hold the ultrasonic probe to scan the calibration tooling. In this way, n first images can be obtained.

[0160] Those skilled in the art can understand that in the above method of the specific implementation manner, the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined according to its function and possible internal logic.

[0161] If the technical solution of this application involves personal information, the product using the technical solution of this application has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using the technical solution of this application has obtained the individual's separate consent before processing the sensitive personal information, and at the same time meets the "explicit consent" requirement. For example, on personal information collection devices such as cameras, clear and prominent signs are set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that he or she agrees to the collection of his or her personal information; or on the device that processes personal information, the personal information processing rules are notified by obvious signs / information, and the individual's authorization is obtained through pop-up information or by asking the individual to upload his or her personal information; among which, personal information processing may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

[0162] The method of the embodiment of the present application is described in detail above, and the device of the embodiment of the present application is provided below.

[0163] See also Figure 5 , Figure 5 A schematic diagram of the structure of a device for calibrating image display delay provided in an embodiment of the present application. The device 1 for calibrating image display delay is used to determine the display delay of an image captured by an image acquisition device when a display device displays the image. The device 1 for calibrating image display delay comprises: an acquisition unit 11 and a processing unit 12. Specifically:

[0164] An acquisition unit 11 is used to acquire n first images, where n is an integer greater than 1, and the n first images include n images acquired by the image acquisition device of the target object when there is relative motion between the image acquisition device and the target object;

[0165] The acquisition unit 11 is used to acquire a target display result, wherein the target display result includes a display result obtained by displaying the n first images on a display device in the acquisition order of the n first images;

[0166] A processing unit 12, configured to obtain a first relationship based on the position of the target object in the n first images, wherein the first relationship represents a relationship between the position of the target object in the images captured by the image acquisition device and time;

[0167] The processing unit 12 is used to obtain a second relationship based on the position of the target object in the target display result, wherein the second relationship represents a relationship between the position of the target object in the image displayed by the display device and time;

[0168] The processing unit 12 is configured to obtain the display delay of the display device based on the time difference between the first relationship and the second relationship.

[0169] Combined with any implementation manner of the present application, the processing unit 12 is specifically configured to:

[0170] Determine a first position corresponding to a first time from the first relationship;

[0171] Determine at least one candidate time corresponding to the first position from the second relationship;

[0172] Determine a time with the smallest difference from the first time among the at least one candidate time as a second time;

[0173] Obtain the display delay according to the time difference between the first time and the second time.

[0174] Combined with any implementation manner of the present application, the processing unit 12 is further configured to:

[0175] Based on the first relationship, determine a first curve that matches the first relationship;

[0176] Determine a point with a slope greater than or equal to a slope threshold in the first curve as a reference point;

[0177] Determine the time corresponding to the reference point in the first relationship as the first time.

[0178] Combined with any implementation manner of the present application, the n first images include n images collected by the image acquisition device when the target object is moving;

[0179] The processing unit 12 is further configured to:

[0180] Obtain a third relationship, where the third relationship is the relationship between the position of the target object in the world coordinate system and time;

[0181] Use a first conversion relationship to convert the position in the display result obtained from the n first images of the target object to the world coordinate system to obtain n converted positions, where the first conversion relationship is used to convert the position in the display result of the display device to the position in the world coordinate system;

[0182] Based on the n converted positions and the display time of the display results of the n first images, obtain a fourth relationship, where the fourth relationship is the relationship between the position of the target object in the world coordinate system and time;

[0183] The processing unit 12 is specifically configured to:

[0184] Determine a first time delay between the first relationship and the third relationship;

[0185] Determine a second time delay between the fourth relationship and the third relationship;

[0186] Based on the difference between the second time delay and the first time delay, obtain the display time delay.

[0187] Combined with any implementation manner of the present application, the processing unit 12 is specifically configured to:

[0188] Determine the time difference between the first time delay and the second time delay as the display time delay.

[0189] Combined with any implementation manner of the present application, the processing unit is specifically configured to:

[0190] Based on the first relationship, determine a first curve that matches the first relationship;

[0191] Based on the third relationship, determine a third curve that matches the third relationship;

[0192] Intercept a curve with a time span of a preset value from the first curve as the fourth curve;

[0193] Determine a fifth curve that matches the fourth curve from the third curve;

[0194] Based on the time delay between the fourth curve and the fifth curve, obtain the first time delay.

[0195] In the embodiments of the present application, the n first images include n images collected by an image acquisition device for a target object in the case where there is relative motion between the image acquisition device and the target object. After obtaining the n first images, the calibration device obtains a first relationship based on the position of the target object in the n first images, where the first relationship characterizes the relationship between the position of the target object in the image collected by the image acquisition device and time. The target display result includes the display result obtained by displaying the n first images on a display device in the acquisition order of the n first images. After obtaining the target result, the calibration device obtains a second relationship based on the position of the target object in the target display result, where the second relationship characterizes the relationship between the position of the target object in the image displayed by the display device and time. Finally, based on the time difference between the first relationship and the second relationship, the display time delay of the display device can be obtained.

[0196] In some embodiments, the functions or modules included in the device provided in the embodiments of the present application can be used to execute the methods described in the method embodiments above. The specific implementation can refer to the description of the method embodiments above. For the sake of brevity, it will not be repeated here.

[0197] Figure 6 This is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present application. The electronic device 2 includes a processor 21 and a memory 22. Optionally, the electronic device 2 further includes an input device 23 and an output device 24. The processor 21, the memory 22, the input device 23, and the output device 24 are coupled through a connector, which includes various interfaces, transmission lines, buses, etc., and the embodiments of the present application do not limit this. It should be understood that in various embodiments of the present application, coupling means being interconnected in a specific manner, including being directly connected or indirectly connected through other devices. For example, they can be connected through various interfaces, transmission lines, buses, etc.

[0198] The processor 21 can be one or more graphics processing units (GPUs). When the processor 21 is a single GPU, the GPU can be a single-core GPU or a multi-core GPU. Optionally, the processor 21 can be a processor group composed of multiple GPUs, and multiple processors are coupled to each other through one or more buses. Optionally, the processor can also be other types of processors, etc., and the embodiments of the present application do not limit this.

[0199] The memory 22 can be used to store computer program instructions and various computer program codes including the program codes for executing the solution of the present application. Optionally, the memory includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or compact disc read-only memory (CD-ROM), and this memory is used for relevant instructions and data.

[0200] The input device 23 is used to input data and / or signals, and the output device 24 is used to output data and / or signals. The input device 23 and the output device 24 can be independent devices or an integrated device.

[0201] It can be understood that in the embodiments of the present application, the memory 22 can not only be used to store relevant instructions, but also be used to store relevant data, and the embodiments of the present application do not limit the specific data stored in this memory.

[0202] It can be understood that Figure 6A simplified design of an electronic device is merely shown. In practical applications, the electronic device may also separately include other necessary components, including but not limited to any number of input / output devices, processors, memories, etc., and all electronic devices that can implement the embodiments of the present application are within the protection scope of the present application.

[0203] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0204] Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments, and will not be repeated here. Those skilled in the art can also clearly understand that each embodiment of the present application has different emphases. For the convenience and conciseness of description, the same or similar parts may not be repeated in different embodiments. Therefore, parts not described or not described in detail in a certain embodiment can be referred to the descriptions of other embodiments.

[0205] In several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the couplings, direct couplings, or communication connections shown or discussed with each other can be through some interfaces, and the indirect couplings or communication connections of the devices or units can be in an electrical, mechanical, or other form.

[0206] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they can be located in one place, or can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0207] In addition, the functional units in each embodiment of the present application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0208] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center in a wired manner (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wirelessly (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more integrated available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital versatile disc (DVD)), or a semiconductor medium (e.g., solid state disk (SSD)), etc.

[0209] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware with a computer program. The program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above method embodiments. The foregoing storage medium includes: various media that can store program codes such as read-only memory (ROM) or random access memory (RAM), magnetic disks, or optical discs.

Claims

1. A method for calibrating image display delay, characterized in that: The method is used to determine the display delay of a display device displaying an image acquired by an image acquisition device, and the method comprises: Acquire n first images, where n is an integer greater than 1, the n first images include n images of the target object acquired by the image acquisition device when there is relative motion between the image acquisition device and the target object, the n first images include n images acquired by the image acquisition device when the target object moves, the image acquisition device includes an ultrasonic acquisition device, and the first image is an ultrasonic image; Acquiring a target display result, the target display result comprising a display result obtained by displaying the n first images on a display device in the order in which the n first images are acquired, after acquiring images using the ultrasound acquisition device, the ultrasound acquisition device transmits the acquired ultrasound images to the display device via a communication connection, so that the ultrasound images acquired by the ultrasound acquisition device are displayed via the display device; Based on the position of the target object in the n first images, a first relationship is obtained, wherein the first relationship represents the relationship between the position of the target object in the images captured by the image acquisition device and time; Based on the position of the target object in the target display result, a second relationship is obtained, where the second relationship represents the relationship between the position of the target object in the image displayed by the display device and time; Acquire a third relationship, where the third relationship is a relationship between the position and time of the target object in a world coordinate system; Using a first conversion relationship, converting the position of the target object in the display result obtained by the n first images to the world coordinate system to obtain n converted positions, wherein the first conversion relationship is used to convert the position in the display result of the display device to the position in the world coordinate system; Based on the n converted positions and the display time of the display results of the n first images, a fourth relationship is obtained, where the fourth relationship is the relationship between the position and time of the target object in the world coordinate system; Based on the time difference between the first relationship and the second relationship, the display delay of the display device is obtained, and the display delay is the time difference between the ultrasonic acquisition device acquiring the ultrasonic image and the display device displaying the ultrasonic image acquired by the ultrasonic acquisition device; The method of obtaining the display delay of the display device based on the time difference between the first relationship and the second relationship includes: determining a first delay between the first relationship and the third relationship; determining a second delay between the fourth relationship and the third relationship; and obtaining the display delay based on the difference between the second delay and the first delay.

2. The method according to claim 1, characterized in that: The obtaining the display delay based on the time difference between the second delay and the first delay includes: A time difference between the first delay and the second delay is determined as the display delay.

3. The method according to claim 1, characterized in that The determining a first time delay between the first relationship and the third relationship includes: Based on the first relationship, determining a first curve matching the first relationship; Based on the third relationship, determining a third curve matching the third relationship; Extracting a curve with a time span of a preset value from the first curve as a fourth curve; determining a fifth curve from the third curve that matches the fourth curve; The first time delay is obtained based on the time delay between the fourth curve and the fifth curve.

4. A device for calibrating image display delay, characterized in that: The image display delay calibration device is used to determine the display delay of the display device displaying the image acquired by the image acquisition device, and the image display delay calibration device includes: an acquisition unit, configured to acquire n first images, where n is an integer greater than 1, and the n first images include n images acquired by the image acquisition device of the target object when there is relative motion between the image acquisition device and the target object, and the n first images include n images acquired by the image acquisition device when the target object moves, the image acquisition device includes an ultrasonic acquisition device, and the first image is an ultrasonic image; The acquisition unit is used to acquire a target display result, wherein the target display result includes a display result obtained by displaying the n first images on a display device in the acquisition order of the n first images, and after the ultrasound acquisition device acquires an image, the ultrasound acquisition device transmits the acquired ultrasound image to the display device through a communication connection, so that the ultrasound image acquired by the ultrasound acquisition device is displayed through the display device; A processing unit, configured to obtain a first relationship based on the position of the target object in the n first images, wherein the first relationship represents a relationship between the position of the target object in the images captured by the image acquisition device and time; The processing unit is used to obtain a second relationship based on the position of the target object in the target display result, wherein the second relationship represents a relationship between the position of the target object in the image displayed by the display device and time; The acquisition unit is used to acquire a third relationship, where the third relationship is a relationship between the position and time of the target object in the world coordinate system; The processing unit is used to convert the position of the target object in the display result obtained by the n first images into the world coordinate system by using a first conversion relationship to obtain n converted positions, wherein the first conversion relationship is used to convert the position in the display result of the display device into a position in the world coordinate system; The processing unit is used to obtain a fourth relationship based on the n converted positions and the display time of the display results of the n first images, wherein the fourth relationship is a relationship between the position and time of the target object in the world coordinate system; The processing unit is configured to obtain the display delay of the display device based on the time difference between the first relationship and the second relationship, wherein the display delay is the time difference between the ultrasonic acquisition device acquiring the ultrasonic image and the display device displaying the ultrasonic image acquired by the ultrasonic acquisition device; The method of obtaining the display delay of the display device based on the time difference between the first relationship and the second relationship includes: determining a first delay between the first relationship and the third relationship; determining a second delay between the fourth relationship and the third relationship; and obtaining the display delay based on the difference between the second delay and the first delay.

5. An electronic device, characterized in that: include: A processor and a memory, wherein the memory is used to store computer program codes, wherein the computer program codes include computer instructions, and when the processor executes the computer instructions, the electronic device executes the method according to any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions. When the program instructions are executed by a processor, the processor is caused to execute the method according to any one of claims 1 to 3.

7. A computer program product, characterized in that The computer program product comprises a computer program; when the computer program is run on a computer, the computer is caused to execute the method according to any one of claims 1 to 3.

Images