Image display time delay calibration method and related product

Abstract

The invention discloses an image display time delay calibration method and a related product. The method is used for determining the display time delay when a display device displays an image collected by an image collection device, and comprises the steps that n first images are obtained, n is an integer larger than 1, and the n first images comprise n images collected by the image collection device for a target object under the condition that relative movement exists between the image collection device and the target object; obtaining a target display result, wherein the target display result comprises a display result obtained by displaying the n first images on the display device according to a collection sequence of the n first images; obtaining a first relation based on the positions of the target object in the n first images; obtaining a second relationship based on the position of the target object in the target display result; and obtaining the display time delay of the display device based on the time difference between the first relationship and the second relationship.

Description

Technical Field

[0001] The present application relates to the field of medical image processing technology, and in particular to a method for calibrating image display delay and related products. Background Art

[0002] There is a communication connection between the ultrasonic acquisition device and the display device. After the ultrasonic acquisition device acquires an image, the ultrasonic acquisition device transmits the acquired ultrasonic image to the display device through the communication connection, so that the ultrasonic image is displayed on the display device. In this process, there is a time difference between the ultrasonic acquisition device acquiring the ultrasonic image and the display device displaying the ultrasonic image, and this time difference is the display delay. Therefore, how to determine the display delay is of great significance. Summary of the invention

[0003] The present application provides a method for calibrating image display delay and related products to determine the display delay of a display device displaying an image captured by an image acquisition device, wherein the related products include an image display delay calibration device, an electronic device, a computer-readable storage medium, and a computer program product.

[0004] In a first aspect, a method for calibrating image display delay is provided, the method being used to determine the display delay of an image captured by an image acquisition device when a display device displays the image, the method comprising: Acquire n first images, where n is an integer greater than 1, and the n first images include n images of the target object acquired by the image acquisition device when there is relative motion between the image acquisition device and the target object; Acquire a target display result, wherein the target display result includes a display result obtained by displaying the n first images on a display device in the order in which the n first images are acquired; Based on the position of the target object in the n first images, a first relationship is obtained, wherein the first relationship represents the relationship between the position of the target object in the images captured by the image acquisition device and time; Based on the position of the target object in the target display result, a second relationship is obtained, where the second relationship represents the relationship between the position of the target object in the image displayed by the display device and time; The display delay of the display device is obtained based on the time difference between the first relationship and the second relationship.

[0005] In combination with any implementation manner of the present application, obtaining the display delay of the display device based on the time difference between the first relationship and the second relationship includes: determining a first position corresponding to a first time from the first relationship; determining at least one candidate time corresponding to the first position from the second relationship; Determine, from the at least one candidate time, a time having the smallest difference from the first time as the second time; The display delay is obtained according to the time difference between the first time and the second time.

[0006] In combination with any implementation manner of the present application, before determining the first position corresponding to the first time from the first relationship, the method further includes: Based on the first relationship, determining a first curve matching the first relationship; Determine a point from the first curve whose slope is greater than or equal to a slope threshold as a reference point; A time corresponding to the reference point in the first relationship is determined as the first time.

[0007] In combination with any embodiment of the present application, the n first images include n images captured by the image capture device when the target object is moving; Before obtaining the display delay of the display device based on the time difference between the first relationship and the second relationship, the method further includes: Acquire a third relationship, where the third relationship is a relationship between the position and time of the target object in a world coordinate system; Using a first conversion relationship, converting the position of the target object in the display result obtained by the n first images to the world coordinate system to obtain n converted positions, wherein the first conversion relationship is used to convert the position in the display result of the display device to the position in the world coordinate system; Based on the n converted positions and the display time of the display results of the n first images, a fourth relationship is obtained, where the fourth relationship is the relationship between the position and time of the target object in the world coordinate system; The obtaining the display delay of the display device based on the time difference between the first relationship and the second relationship includes: determining a first time delay between the first relationship and the third relationship; determining a second time delay between the fourth relationship and the third relationship; The display delay is obtained based on the difference between the second delay and the first delay.

[0008] In combination with any implementation manner of the present application, obtaining the display delay based on the time difference between the first delay and the second delay includes: A time difference between the first delay and the second delay is determined as the display delay.

[0009] In combination with any implementation manner of this application, The determining a first time delay between the first relationship and the third relationship includes: Based on the first relationship, determining a first curve matching the first relationship; Based on the third relationship, determining a third curve matching the third relationship; Extracting a curve with a time span of a preset value from the first curve as a fourth curve; determining a fifth curve from the third curve that matches the fourth curve; The first time delay is obtained based on the time delay between the fourth curve and the fifth curve.

[0010] In a second aspect, a device for calibrating image display delay is provided, wherein the device for calibrating image display delay is used to determine the display delay of a display device displaying an image acquired by an image acquisition device, and the device for calibrating image display delay comprises: an acquisition unit, configured to acquire n first images, where n is an integer greater than 1, and the n first images include n images acquired by the image acquisition device of the target object when there is relative motion between the image acquisition device and the target object; The acquisition unit is used to acquire a target display result, wherein the target display result includes a display result obtained by displaying the n first images on a display device in the acquisition order of the n first images; A processing unit, configured to obtain a first relationship based on the position of the target object in the n first images, wherein the first relationship represents a relationship between the position of the target object in the images captured by the image acquisition device and time; The processing unit is used to obtain a second relationship based on the position of the target object in the target display result, wherein the second relationship represents a relationship between the position of the target object in the image displayed by the display device and time; The processing unit is configured to obtain the display delay of the display device based on a time difference between the first relationship and the second relationship.

[0011] In conjunction with any implementation manner of the present application, the processing unit is specifically used to: determining a first position corresponding to a first time from the first relationship; determining at least one candidate time corresponding to the first position from the second relationship; Determine, from the at least one candidate time, a time having the smallest difference from the first time as the second time; The display delay is obtained according to the time difference between the first time and the second time.

[0012] In combination with any embodiment of the present application, the processing unit is further used for: Based on the first relationship, determining a first curve matching the first relationship; Determine a point from the first curve whose slope is greater than or equal to a slope threshold as a reference point; A time corresponding to the reference point in the first relationship is determined as the first time.

[0013] In combination with any embodiment of the present application, the n first images include n images captured by the image capture device when the target object is moving; The processing unit is further used for: Acquire a third relationship, where the third relationship is a relationship between the position and time of the target object in a world coordinate system; Using a first conversion relationship, converting the position of the target object in the display result obtained by the n first images to the world coordinate system to obtain n converted positions, wherein the first conversion relationship is used to convert the position in the display result of the display device to the position in the world coordinate system; Based on the n converted positions and the display time of the display results of the n first images, a fourth relationship is obtained, where the fourth relationship is the relationship between the position and time of the target object in the world coordinate system; The processing unit is specifically used for: Based on the first relationship, determining a first curve matching the first relationship; Based on the third relationship, determining a third curve matching the third relationship; Extracting a curve with a time span of a preset value from the first curve as a fourth curve; determining a fifth curve from the third curve that matches the fourth curve; The first time delay is obtained based on the time delay between the fourth curve and the fifth curve.

[0014] In conjunction with any implementation manner of the present application, the processing unit is specifically used to: A time difference between the first delay and the second delay is determined as the display delay.

[0015] In conjunction with any implementation manner of the present application, the processing unit is specifically used to: Based on the second relationship, determining a second curve, where the second curve is a curve matching the second relationship; Based on the third relationship, determining a third curve, wherein the third curve is a curve matching the third relationship; Extracting a curve with a time span of a preset value from the second curve as a fourth curve; determining a fifth curve from the third curve that matches the fourth curve; The second time delay is obtained based on the time delay between the fourth curve and the fifth curve.

[0016] In a third aspect, an electronic device is provided, comprising: a processor and a memory, the memory being used to store computer program code, the computer program code comprising computer instructions, and when the processor executes the computer instructions, the electronic device executes the method as described in the first aspect above and any possible implementation thereof.

[0017] In a fourth aspect, another electronic device is provided, comprising: a processor, a sending device, an input device, an output device and a memory, wherein the memory is used to store computer program code, and the computer program code includes computer instructions. When the processor executes the computer instructions, the electronic device executes the method as described in the first aspect above and any possible implementation method thereof.

[0018] In a fifth aspect, a computer-readable storage medium is provided, in which a computer program is stored. The computer program includes program instructions, and when the program instructions are executed by a processor, the processor is caused to execute the method as described in the first aspect above and any possible implementation method thereof.

[0019] In a sixth aspect, a computer program product is provided, which includes a computer program or instructions, and when the computer program or instructions are run on a computer, the computer is enabled to execute the method of the above-mentioned first aspect and any possible implementation thereof.

[0020] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application.

[0021] In the present application, the n first images include n images of the target object captured by the image acquisition device when there is relative motion between the image acquisition device and the target object. After acquiring the n first images, the calibration device obtains a first relationship based on the position of the target object in the n first images, wherein the first relationship represents the relationship between the position of the target object in the image captured by the image acquisition device and time. The target display result includes a display result obtained by displaying the n first images on the display device in the acquisition order of the n first images. After acquiring the target result, the calibration device obtains a second relationship based on the position of the target object in the target display result, wherein the second relationship represents the relationship between the position of the target object in the image displayed by the display device and time. Finally, based on the time difference between the first relationship and the second relationship, the display delay of the display device can be obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the background technology, the drawings required for use in the embodiments of the present application or the background technology will be described below.

[0023] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.

[0024] Figure 1 A schematic diagram of a flow chart of a method for calibrating ultrasound image display delay provided in an embodiment of the present application; Figure 2 A schematic diagram of a first relationship and a second relationship before calibration provided in an embodiment of the present application; Figure 3 A schematic diagram of a calibrated first relationship and a second relationship provided in an embodiment of the present application; Figure 4 A schematic diagram of a scenario of scanning a calibration tool using an ultrasonic probe provided in an embodiment of the present application; Figure 5 A schematic diagram of the structure of a device for calibrating image display delay provided in an embodiment of the present application; Figure 6 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0026] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0027] Mentioning "embodiment" in this article means that the specific features, structures or characteristics described in conjunction with the embodiment may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments. It should be understood that in this application, "at least one (item)" means one or more, "multiple" means two or more, and "at least two (items)" means two or three and more.

[0028] The execution subject of the embodiment of the present application is a calibration device for ultrasonic image display delay (hereinafter referred to as the calibration device), wherein the calibration device can be any electronic device that can execute the technical solution disclosed in the embodiment of the method of the present application. The calibration device can determine the display delay of the ultrasonic image collected by the ultrasonic acquisition device on the display device by performing the calibration of the ultrasonic image display delay. Specifically, there is a communication connection between the ultrasonic acquisition device and the display device. After the ultrasonic acquisition device is used to collect the image, the ultrasonic acquisition device transmits the collected ultrasonic image to the display device through the communication connection to display the ultrasonic image through the display device. In this process, there is a time difference between the ultrasonic acquisition device collecting the ultrasonic image and the display device displaying the ultrasonic image. The time difference is the display delay. For example, the ultrasonic acquisition device collects the ultrasonic image at time t1, and the display device displays the ultrasonic image at time t2. At this time, the display delay is t2-t1. Optionally, the calibration device can be one of the following: a mobile phone, a computer, a tablet computer, and a wearable smart device.

[0029] It should be understood that the method embodiment of the present application can also be implemented by a processor executing a computer program code. The following describes the embodiment of the present application in conjunction with the drawings in the embodiment of the present application. Figure 1 , Figure 1 A flowchart of a method for calibrating ultrasound image display delay provided in an embodiment of the present application.

[0030] 101. Acquire n first ultrasound images.

[0031] In the embodiment of the present application, n is an integer greater than 1, and the n first images include n images of the target object captured by the image capture device when there is relative motion between the image capture device and the target object. The relative motion between the image capture device and the target object may be that the target object is stationary and the image capture device is in motion. The relative motion between the image capture device and the target object may be that the target object is in motion and the image capture device is stationary. The relative motion between the image capture device and the target object may be that both the target object and the image capture device are in motion. The n first images are all captured by the image capture device, and the capture time of any two first images is different.

[0032] In the embodiment of the present application, the target object can be any object, for example, the target object is a line, and for example, the target object is a bump. The imaging device can be any imaging device, and optionally, the imaging device is an ultrasound probe, and the first image is an ultrasound image.

[0033] In an implementation of acquiring n first images, a calibration device receives n first images input by a user through an input component, wherein the input component includes: a mouse, a keyboard, a touch screen, a touch pad, and an audio input device.

[0034] In another implementation method of acquiring n first images, the calibration device receives n first images sent by a terminal, wherein the terminal includes: a mobile phone, a computer, a tablet computer, and a smart wearable device.

[0035] 102. Get the target display result.

[0036] In an embodiment of the present application, the target display result includes a display result obtained by displaying n first images on a display device in the order in which the n first images are acquired. Specifically, the display device displays n first images in the order in which the n first images are acquired, and the display result of the n first images can be obtained, which is the target display result. For example, the n first images include image a and image b, wherein the acquisition time of image a is earlier than the acquisition time of image b, then the acquisition order of the n first images is to first acquire image a and then acquire image b. When the display device displays the n first images, it first displays image a and then displays image b. At this time, the target display result includes the display result of image a and the display result of image b. Optionally, the display result of the display device displaying the first image is a display image, and the target display result includes n display images corresponding to the n first images. For example, the n first images include image a and image b, wherein the display image corresponding to image a is display image c, and the display image corresponding to image b is display image d, and at this time the target display result includes display image c and display image d.

[0037] Optionally, there is a communication connection between the image acquisition device and the display device, and after acquiring the first image, the image acquisition device sends the first image to the display device through the communication connection, so that the display device displays the first image. Optionally, the display device is a device including a display device, for example, the display device is a computer including a display, wherein the display is the display device.

[0038] In an implementation of acquiring a target display result, a calibration device receives a target display result input by a user through an input component.

[0039] In another implementation of acquiring the target display result, the calibration device receives the target display result sent by the terminal.

[0040] 103. Obtain a first relationship based on positions of the target object in the n first images.

[0041] In an embodiment of the present application, the first relationship characterizes the relationship between the position of the target object in the image captured by the image acquisition device and the time, that is, based on the first relationship, the position of the target object in the image captured by the image acquisition device at any time can be determined. In one possible implementation method, the calibration device determines the position of the target object in n first images and obtains n imaging positions. Based on the n imaging positions and the acquisition times corresponding to the n first images, n imaging data points are obtained, wherein the imaging data points include the imaging position and the acquisition time corresponding to the imaging position. The n imaging data points are fitted to obtain the first relationship.

[0042] 104. Obtain a second relationship based on the position of the target object in the target display result.

[0043] In an embodiment of the present application, the second relationship characterizes the relationship between the position of the target object in the image displayed by the display device and the time, that is, based on the second relationship, the position of the target object in the image displayed by the display device at any time can be determined. In one possible implementation method, the calibration device determines the position of the target object in the target display result and obtains n display positions. Based on the n display positions and the display time of the n first images, n display data points are obtained, wherein the display data points include the display position and the display time corresponding to the display position. The n display data points are fitted to obtain the second relationship.

[0044] 105. Obtain the display delay of the display device based on the time difference between the first relationship and the second relationship.

[0045] Considering that if there is a time delay between the time when the image acquisition device acquires the image and the time when the display device displays the image, the first relationship should be different from the second relationship. Specifically, there is a time difference between the first relationship and the second relationship. Therefore, the calibration device can obtain the display delay of the display device based on the time difference between the first relationship and the second relationship.

[0046] In one possible implementation, the calibration device uses the time difference between the first relationship and the second relationship as the display delay of the display device. In another possible implementation, the calibration device determines the time difference between the first relationship and the second relationship, determines the sum of the time difference and a first preset value to obtain the display delay, wherein the first preset value is a positive number. In yet another possible implementation, the calibration device determines the time difference between the first relationship and the second relationship, determines the product of the time difference and a second preset value to obtain the display delay, wherein the second preset value is a positive number.

[0047] In an embodiment of the present application, the n first images include n images of the target object captured by the image acquisition device when there is relative motion between the image acquisition device and the target object. After acquiring the n first images, the calibration device obtains a first relationship based on the position of the target object in the n first images, wherein the first relationship represents the relationship between the position of the target object in the image captured by the image acquisition device and time. The target display result includes a display result obtained by displaying the n first images on the display device in the acquisition order of the n first images. After acquiring the target result, the calibration device obtains a second relationship based on the position of the target object in the target display result, wherein the second relationship represents the relationship between the position of the target object in the image displayed by the display device and time. Finally, based on the time difference between the first relationship and the second relationship, the display delay of the display device can be obtained.

[0048] As an optional implementation manner, the calibration device performs the following steps during the process of executing step 105: 201. Determine a first position corresponding to a first time from the first relationship.

[0049] In the embodiment of the present application, the first time is any time in the first relationship. In the first relationship, positions correspond to times one by one, and the position corresponding to the first time is the first position.

[0050] 202. Determine at least one candidate time corresponding to the first position from the second relationship.

[0051] Since in the second relationship, there is at least one different time corresponding to the same position, for example, in the second relationship, the position corresponding to time t1 and the position corresponding to time t2 are both p, that is, the position of the target in the image displayed by the display device at time t1 is the same as the position of the target in the image displayed by the display device at time t2. Therefore, the calibration device can determine at least one time corresponding to the first position from the second relationship, that is, at least one candidate time.

[0052] 203. Determine, from the at least one candidate time, a time that has the smallest difference from the first time as the second time.

[0053] Considering that the position of the target object in the first image is usually the same as the position of the target object in the displayed image of the first image, and there is a time corresponding to the position of the target object in the first image in the first relationship, and there is a time corresponding to the position of the target object in the displayed image in the second relationship, for the convenience of expression, the time corresponding to the position of the target object in the first image in the first relationship and the time corresponding to the position of the target object in the displayed image in the second relationship are referred to as times with a corresponding relationship. For example, the position of the target object in the first image a is the first position, wherein the acquisition time of the first image a is t1. In the first relationship, the time corresponding to the first position is t1. The display device displays the first image a at time t2 to obtain the displayed image b. At this time, the position of the target object in the displayed image b is also the first position, and the time corresponding to the first position in the second relationship is t2. At this time, t1 and t2 are times with a corresponding relationship.

[0054] Since the time corresponds to the position in the first relationship and the second relationship, the calibration device uses the position as a bridge to determine the time with a corresponding relationship from the first relationship and the second relationship. Therefore, after determining the first position corresponding to the first time from the first relationship, the calibration device can further use the first position as a bridge to determine the time with a corresponding relationship with the first time from the second relationship. Therefore, after determining at least one candidate time from the second relationship, the calibration device can further determine the time with a corresponding relationship with the first time from the at least one candidate time.

[0055] When the number of candidate times is 1, the calibration device determines that the candidate time is the time that has a corresponding relationship with the first time, that is, the second time. When the number of candidate times is 2, it is necessary to determine a time that has a corresponding relationship with the first time from at least two candidate times. Since the display delay of the display device is usually small, the time difference between the two times that have a corresponding relationship in the first relationship and the second relationship is small, so the calibration device determines the time with the smallest difference from the first time from at least one candidate time as the time that has a corresponding relationship with the first time, that is, the second time.

[0056] 204. Obtain the display delay according to a time difference between the first time and the second time.

[0057] In one possible implementation, the calibration device determines the time difference between the first time and the second time to obtain the display delay. In another possible implementation, the calibration device determines the time difference between the first time and the second time. Determine the sum of the time difference and a first preset value to obtain the display delay. In yet another possible implementation, the calibration device determines the time difference between the first time and the second time. Determine the product of the time difference and the first preset value to obtain the display delay.

[0058] As an optional implementation manner, the calibration device further performs the following steps before performing step 201: 301. Based on the first relationship, determine a first curve matching the first relationship.

[0059] 302. Determine, from the first curve, a point whose slope is greater than or equal to a slope threshold as a reference point.

[0060] 303. Determine the time corresponding to the reference point in the first relationship as the first time.

[0061] The slope of a point in the first curve is large, indicating that the position corresponding to the point has a large change. Therefore, before step 201, the point is used as a reference point, and the time corresponding to the point is used as the first time. Then, through steps 201 to 204, the first position corresponding to the first time is determined, and the display delay is obtained by determining the time difference between the time when the imaging device captures the first image corresponding to the first position and the time when the display device displays the first image corresponding to the first position. The display delay can be obtained by determining the time difference between the time when the imaging device captures the first image corresponding to the position with a large change and the time when the display device displays the first image corresponding to the position with a large change. This can improve the accuracy of the display delay.

[0062] As an optional implementation, the n first images include n images captured by an image acquisition device when the target object is moving. Optionally, the image acquisition device is in a stationary state during the process of capturing the n first images. In this implementation, before executing step 105, the calibration device further performs the following steps: 401. Obtain the third relationship.

[0063] In the embodiment of the present application, the third relationship is the relationship between the position of the target object in the world coordinate system and the time. That is, based on the third relationship, the position of the target object in the world coordinate system at any time can be determined. In one possible implementation method, the calibration device determines m motion positions of the target object during the motion process. Based on the m motion positions and the motion times corresponding to the m motion positions, m motion data points are obtained, wherein the motion data points include the motion positions and the motion times corresponding to the motion positions. The n motion data points are fitted to obtain the third relationship.

[0064] 402. Using the first conversion relationship, convert the position of the target object in the display results of the n first images into the world coordinate system to obtain n converted positions.

[0065] In the embodiment of the present application, the first conversion relationship is used to convert the position in the display result of the display device into the position in the world coordinate system. Therefore, the calibration device can use the first conversion relationship to convert the position of the target object in the display result obtained by the n first images into the world coordinate system to obtain n converted positions.

[0066] 403. Obtain a fourth relationship based on the n conversion positions and the display time of the display results obtained from the n first images.

[0067] In the embodiment of the present application, the fourth relationship is the relationship between the position of the target object in the world coordinate system and the time, that is, the position of the target object in the world coordinate system at any time can be determined based on the fourth relationship. It should be understood that since the n conversion positions are obtained based on the position of the target object in the display results of the n first images, and the position of the target object in the display results of the first image is delayed compared to the position of the target object in the world coordinate system, there is a time delay between the third relationship and the fourth relationship. In a possible implementation method, the calibration device obtains n conversion data points based on the n conversion positions and the display time of the n first images, wherein the conversion data points include the conversion position and the display time corresponding to the conversion position. The n conversion data points are fitted to obtain the fourth relationship.

[0068] After obtaining the fourth relationship, the calibration device performs the following steps during the execution of step 105: 404. Determine a first time delay between the first relationship and the third relationship.

[0069] In one possible implementation, the calibration device determines the time difference between the first relationship and the third relationship to obtain the first time delay.

[0070] 405. Determine a second time delay between the fourth relationship and the third relationship.

[0071] In one possible implementation, the calibration device determines the time difference between the third relationship and the fourth relationship to obtain the second time delay.

[0072] 406. Obtain the display delay based on a difference between the second delay and the first delay.

[0073] In one possible implementation, the calibration device determines the difference between the first delay and the second delay to obtain the display delay. In another possible implementation, the calibration device determines the time difference between the first delay and the second delay. The sum of the time difference and the first preset value is determined to obtain the display delay. In yet another possible implementation, the calibration device determines the time difference between the first delay and the second delay. The product of the time difference and the second preset value is determined to obtain the display delay.

[0074] Considering the display error of the display device, the position of the target in the display result of the first image may be different from the position of the target in the first image. For example, due to the difference in the display resolution of the display device and the resolution of the first image, the position of the target in the display result of the first image may be different from the position of the target in the first image. When this happens, the positions originally corresponding to the same first image will be different in the first relationship and the second relationship. For example, the display result obtained by the display device by displaying the first image a is the display image b. Then in the first relationship, the position corresponding to the first image a is the position of the target in the first image a. In the second relationship, the position corresponding to the display image b is the position of the target in the display image b. Since the display image b is the image corresponding to the first image a, in the second relationship, the position corresponding to the first image a is the position corresponding to the display image b, that is, in the second relationship, the position corresponding to the first image a is the position of the target in the display image b. If the position of the target object in the first image a is position c, and the position of the target object in the displayed image b is position d, then in the first relationship, the position corresponding to the first image a is position c, and in the second relationship, the position corresponding to the first image a is position d. If there is a display error in the display device, then position c is different from position d. If the positions originally corresponding to the same first image are different in the first relationship and the second relationship, it will further lead to a large error in the display delay determined based on the time difference between the first relationship and the second relationship.

[0075] Therefore, in this embodiment, the calibration device first uses the first conversion relationship to convert the position of the target object in the display result obtained by the n first images to the world coordinate system, obtains n conversion positions, and removes the display error of the display device. Then, based on the display time of the display results of the n conversion positions and the n first images, the fourth relationship is obtained. Then, the first time delay between the first relationship and the third relationship is determined, wherein the first time delay represents the time delay of the image acquisition device to acquire the position of the target object in the world coordinate system. For example, at time t1, the position of the target object in the world coordinate system is p1, and the image acquisition device acquires p1 at time t2, and the first time delay is t2-t1. The second time delay between the fourth relationship and the third relationship is determined, wherein the second time delay can represent the time delay of the display device to display the position of the target object in the world coordinate system. For example, at time t1, the position of the target object in the world coordinate system is p1, and the display device displays p1 at time t3, and the second time delay is t3-t1. Since the second delay = the first delay + the display delay, the calibration device obtains the display delay based on the time difference between the second delay and the first delay, thereby reducing the influence of the display error of the display device on the determination of the display delay, thereby improving the accuracy of the display delay.

[0076] As an optional implementation, the calibration device performs the following steps during the execution of step 405: 501. Based on the first relationship, determine a first curve matching the first relationship.

[0077] In the embodiment of the present application, the first curve matches the first relationship, that is, the first relationship can be represented by the first curve.

[0078] 502. Based on the third relationship, determine a third curve matching the third relationship.

[0079] In the embodiment of the present application, the third curve matches the third relationship, that is, the third relationship can be represented by the first curve.

[0080] 503. A curve having a time span of a preset value is intercepted from the first curve as a fourth curve.

[0081] 504. Determine a fifth curve matching the fourth curve from the third curve.

[0082] 505. Obtain the first time delay based on the time delay between the fourth curve and the fifth curve.

[0083] Optionally, the fourth curve matches the fifth curve in that the degree of overlap between the fourth curve and the fifth curve is greater than or equal to an overlap threshold. Optionally, the fourth curve matches the fifth curve in that the difference between the position of the target object corresponding to the inflection point of the fourth curve and the position of the target object corresponding to the inflection point of the fifth curve is less than or equal to an inflection point threshold. For example, the fourth curve includes an inflection point a, and the fifth curve includes an inflection point b, wherein the inflection point a indicates that the position of the target object in the world coordinate system at time t1 is p1, and the inflection point b indicates that the position of the target object in the world coordinate system at time t2 is p2. Then the position of the target object corresponding to the inflection point of the fourth curve is p1, and the position of the target object corresponding to the inflection point of the fifth curve is p2.

[0084] In this embodiment, the data volume of the fourth curve is less than that of the first curve, and the data volume of the fifth curve is less than that of the third curve. Therefore, after the calibration device obtains the fourth curve by executing step 503 and obtains the fifth curve by executing step 504, the first time delay is obtained based on the time delay between the fourth curve and the fifth curve, which can reduce the data processing volume for determining the first time delay between the first curve and the third curve and improve the processing speed.

[0085] On the one hand, considering that the larger the time span of the fourth curve is, the larger the data volume of the fourth curve is, the larger the data volume of the fifth curve is, and accordingly, determining the first delay based on the delay between the fourth curve and the fifth curve can improve the accuracy of the first delay. On the other hand, the larger the time span of the fourth curve is, the larger the data volume of the fourth curve is, the larger the data volume of the fifth curve is, and accordingly, the larger the data processing volume of the first delay is obtained based on the delay between the fourth curve and the fifth curve. Therefore, the specific value of the preset value can be set according to actual needs, and this application does not limit this.

[0086] Optionally, after determining the display delay based on the method described above, the display time of the display device based on the display delay can be calibrated, thereby reducing the display error caused by the display delay. For example, when the display device displays an ultrasonic image acquired by an ultrasonic acquisition device, the difference between the current time and the display delay is determined to be the acquisition time of the ultrasonic image, and the acquisition time is displayed while displaying the ultrasonic image.

[0087] Optionally, after the display time of the display device is calibrated based on the display delay, the delay between the first relationship and the second relationship can be reduced. Figure 2 , Figure 2 A schematic diagram of a first relationship and a second relationship before calibration provided in an embodiment of the present application. Figure 2 In the coordinate system shown, the horizontal axis represents time and the vertical axis represents the position of the target object. Figure 2 In the figure, the solid line represents the first relationship and the dotted line represents the second relationship. Figure 2As shown in Figure 2, there is a time delay between the first relationship and the second relationship before calibration, which is the above-mentioned display delay. Figure 3 , Figure 3 A schematic diagram of a calibrated first relationship and a second relationship provided in an embodiment of the present application. Figure 3 In the coordinate system shown, the horizontal axis represents time and the vertical axis represents the position of the target object. Figure 3 In the figure, the solid line represents the first relationship and the dotted line represents the second relationship. Figure 3 As shown, after calibration, the time delay between the first relationship and the second relationship is reduced compared to the time delay before calibration.

[0088] Optionally, the image acquisition device is an ultrasonic probe, and the target object belongs to the calibration tool, that is, the target object is a structure on the calibration tool. The ultrasonic probe can obtain n first images by scanning the calibration tool.

[0089] Optionally, the design requirements for the calibration fixture are as follows: 1) Ultrasonic probes require a medium when used, and water is the most convenient medium, so the calibration tool can be filled with water; 2) The scanning area of ​​the ultrasonic probe has a certain depth and width, so the calibration tool has a height that matches the depth and a length that matches the width; 3) To facilitate the back-and-forth scanning of the ultrasonic probe, the area scanned by the ultrasonic probe on the calibration tooling should be greater than or equal to the area threshold; 4) Since the resolution of the ultrasound image obtained by scanning with the ultrasound probe is low and many artifacts are generated, a thin wire can be selected as the target object in the calibration tooling. This can reduce the artifacts in the ultrasound image, thereby improving the accuracy of the position of the determined target object in the n first images. Optionally, the diameter of the thin wire is less than 1 mm. Optionally, the thin wire is not easy to break; Optional, Figure 4 A schematic diagram of a scenario in which an ultrasonic probe is used to scan a calibration tooling according to an embodiment of the present application. Figure 4 As shown, the user can hold the ultrasonic probe to scan the calibration tooling, so that n first images can be obtained.

[0090] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0091] If the technical solution of this application involves personal information, the product using the technical solution of this application has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using the technical solution of this application has obtained the individual's separate consent before processing the sensitive personal information, and at the same time meets the "explicit consent" requirement. For example, on personal information collection devices such as cameras, clear and prominent signs are set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that he or she agrees to the collection of his or her personal information; or on the device that processes personal information, the personal information processing rules are notified by obvious signs / information, and the individual's authorization is obtained through pop-up information or by asking the individual to upload his or her personal information; among which, personal information processing may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

[0092] The method of the embodiment of the present application is described in detail above, and the device of the embodiment of the present application is provided below.

[0093] See also Figure 5 , Figure 5 A schematic diagram of the structure of a device for calibrating image display delay provided in an embodiment of the present application. The device 1 for calibrating image display delay is used to determine the display delay of an image captured by an image acquisition device when a display device displays the image. The device 1 for calibrating image display delay comprises: an acquisition unit 11 and a processing unit 12. Specifically: An acquisition unit 11 is used to acquire n first images, where n is an integer greater than 1, and the n first images include n images acquired by the image acquisition device of the target object when there is relative motion between the image acquisition device and the target object; The acquisition unit 11 is used to acquire a target display result, wherein the target display result includes a display result obtained by displaying the n first images on a display device in the acquisition order of the n first images; A processing unit 12, configured to obtain a first relationship based on the position of the target object in the n first images, wherein the first relationship represents a relationship between the position of the target object in the images captured by the image acquisition device and time; The processing unit 12 is used to obtain a second relationship based on the position of the target object in the target display result, wherein the second relationship represents a relationship between the position of the target object in the image displayed by the display device and time; The processing unit 12 is configured to obtain the display delay of the display device based on the time difference between the first relationship and the second relationship.

[0094] In combination with any embodiment of the present application, the processing unit 12 is specifically used for: determining a first position corresponding to a first time from the first relationship; determining at least one candidate time corresponding to the first position from the second relationship; Determine, from the at least one candidate time, a time having the smallest difference from the first time as the second time; The display delay is obtained according to the time difference between the first time and the second time.

[0095] In combination with any embodiment of the present application, the processing unit 12 is further configured to: Based on the first relationship, determining a first curve matching the first relationship; Determine a point from the first curve whose slope is greater than or equal to a slope threshold as a reference point; A time corresponding to the reference point in the first relationship is determined as the first time.

[0096] In combination with any embodiment of the present application, the n first images include n images captured by the image capture device when the target object is moving; The processing unit 12 is further used for: Acquire a third relationship, where the third relationship is a relationship between the position and time of the target object in a world coordinate system; Using a first conversion relationship, converting the position of the target object in the display result obtained by the n first images to the world coordinate system to obtain n converted positions, wherein the first conversion relationship is used to convert the position in the display result of the display device to the position in the world coordinate system; Based on the n converted positions and the display time of the display results of the n first images, a fourth relationship is obtained, where the fourth relationship is the relationship between the position and time of the target object in the world coordinate system; The processing unit 12 is specifically used for: determining a first time delay between the first relationship and the third relationship; determining a second time delay between the fourth relationship and the third relationship; The display delay is obtained based on the difference between the second delay and the first delay.

[0097] In combination with any embodiment of the present application, the processing unit 12 is specifically used for: A time difference between the first delay and the second delay is determined as the display delay.

[0098] In conjunction with any implementation manner of the present application, the processing unit is specifically used to: Based on the first relationship, determining a first curve matching the first relationship; Based on the third relationship, determining a third curve matching the third relationship; Extracting a curve with a time span of a preset value from the first curve as a fourth curve; determining a fifth curve from the third curve that matches the fourth curve; The first time delay is obtained based on the time delay between the fourth curve and the fifth curve.

[0099] In an embodiment of the present application, the n first images include n images of the target object captured by the image acquisition device when there is relative motion between the image acquisition device and the target object. After acquiring the n first images, the calibration device obtains a first relationship based on the position of the target object in the n first images, wherein the first relationship represents the relationship between the position of the target object in the image captured by the image acquisition device and time. The target display result includes a display result obtained by displaying the n first images on the display device in the acquisition order of the n first images. After acquiring the target result, the calibration device obtains a second relationship based on the position of the target object in the target display result, wherein the second relationship represents the relationship between the position of the target object in the image displayed by the display device and time. Finally, based on the time difference between the first relationship and the second relationship, the display delay of the display device can be obtained.

[0100] In some embodiments, the functions or modules included in the device provided in the embodiments of the present application can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.

[0101] Figure 6 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. The electronic device 2 includes a processor 21 and a memory 22. Optionally, the electronic device 2 also includes an input device 23 and an output device 24. The processor 21, the memory 22, the input device 23 and the output device 24 are coupled via a connector, and the connector includes various interfaces, transmission lines or buses, etc., which are not limited in the embodiments of the present application. It should be understood that in each embodiment of the present application, coupling refers to mutual connection in a specific manner, including direct connection or indirect connection through other devices, for example, it can be connected through various interfaces, transmission lines, buses, etc.

[0102] The processor 21 may be one or more graphics processing units (GPUs). When the processor 21 is a GPU, the GPU may be a single-core GPU or a multi-core GPU. Optionally, the processor 21 may be a processor group consisting of multiple GPUs, and the multiple processors are coupled to each other via one or more buses. Optionally, the processor may also be other types of processors, etc., which are not limited in the embodiments of the present application.

[0103] The memory 22 can be used to store computer program instructions and various computer program codes including the program code for executing the program code of the present application. Optionally, the memory includes but is not limited to random access memory (RAM), read-only memory (ROM), erasable programmable read only memory (EPROM), or portable read only memory (CD-ROM), which is used for related instructions and data.

[0104] The input device 23 is used to input data and / or signals, and the output device 24 is used to output data and / or signals. The input device 23 and the output device 24 can be independent devices or an integrated device.

[0105] It can be understood that in the embodiment of the present application, the memory 22 can be used not only to store relevant instructions, but also to store relevant data. The embodiment of the present application does not limit the specific data stored in the memory.

[0106] Understandably, Figure 6 Only a simplified design of an electronic device is shown. In practical applications, the electronic device may also include other necessary components, including but not limited to any number of input / output devices, processors, memories, etc., and all electronic devices that can implement the embodiments of the present application are within the protection scope of the present application.

[0107] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0108] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. Those skilled in the art can also clearly understand that the descriptions of the various embodiments of the present application have different focuses. For the convenience and brevity of description, the same or similar parts may not be repeated in different embodiments. Therefore, for parts not described or not described in detail in a certain embodiment, refer to the records of other embodiments.

[0109] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0110] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0111] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0112] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions may be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a digital versatile disc (DVD)), or a semiconductor medium (eg, a solid state disk (SSD)).

[0113] A person skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by a computer program to instruct the relevant hardware, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above-mentioned method embodiments. The aforementioned storage medium includes: a read-only memory (ROM) or a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

Claims

1. A method for calibrating image display delay, characterized in that: The method is used to determine the display delay of a display device displaying an image acquired by an image acquisition device, and the method comprises: Acquire n first images, where n is an integer greater than 1, and the n first images include n images of the target object acquired by the image acquisition device when there is relative motion between the image acquisition device and the target object; Acquire a target display result, wherein the target display result includes a display result obtained by displaying the n first images on a display device in the order in which the n first images are acquired; Based on the position of the target object in the n first images, a first relationship is obtained, wherein the first relationship represents the relationship between the position of the target object in the images captured by the image acquisition device and time; Based on the position of the target object in the target display result, a second relationship is obtained, where the second relationship represents the relationship between the position of the target object in the image displayed by the display device and time; The display delay of the display device is obtained based on the time difference between the first relationship and the second relationship.

2. The method according to claim 1, characterized in that The obtaining the display delay of the display device based on the time difference between the first relationship and the second relationship includes: determining a first position corresponding to a first time from the first relationship; determining at least one candidate time corresponding to the first position from the second relationship; Determine, from the at least one candidate time, a time having the smallest difference from the first time as the second time; The display delay is obtained according to the time difference between the first time and the second time.

3. The method according to claim 2, characterized in that Before determining the first position corresponding to the first time from the first relationship, the method further includes: Based on the first relationship, determining a first curve matching the first relationship; Determine a point from the first curve whose slope is greater than or equal to a slope threshold as a reference point; A time corresponding to the reference point in the first relationship is determined as the first time.

4. The method according to claim 1, characterized in that: The n first images include n images captured by the image acquisition device when the target object is moving; Before obtaining the display delay of the display device based on the time difference between the first relationship and the second relationship, the method further includes: Acquire a third relationship, where the third relationship is a relationship between the position and time of the target object in a world coordinate system; Using a first conversion relationship, converting the position of the target object in the display result obtained by the n first images to the world coordinate system to obtain n converted positions, wherein the first conversion relationship is used to convert the position in the display result of the display device to the position in the world coordinate system; Based on the n converted positions and the display time of the display results of the n first images, a fourth relationship is obtained, where the fourth relationship is the relationship between the position and time of the target object in the world coordinate system; The obtaining the display delay of the display device based on the time difference between the first relationship and the second relationship includes: determining a first time delay between the first relationship and the third relationship; determining a second time delay between the fourth relationship and the third relationship; The display delay is obtained based on the difference between the second delay and the first delay.

5. The method according to claim 4, characterized in that The obtaining the display delay based on the time difference between the first delay and the second delay includes: A time difference between the first delay and the second delay is determined as the display delay.

6. The method according to claim 4, characterized in that The determining a first time delay between the first relationship and the third relationship includes: Based on the first relationship, determining a first curve matching the first relationship; Based on the third relationship, determining a third curve matching the third relationship; Extracting a curve with a time span of a preset value from the first curve as a fourth curve; determining a fifth curve from the third curve that matches the fourth curve; The first time delay is obtained based on the time delay between the fourth curve and the fifth curve.

7. A device for calibrating image display delay, characterized in that: The image display delay calibration device is used to determine the display delay of the display device displaying the image acquired by the image acquisition device, and the image display delay calibration device includes: an acquisition unit, configured to acquire n first images, where n is an integer greater than 1, and the n first images include n images acquired by the image acquisition device of the target object when there is relative motion between the image acquisition device and the target object; The acquisition unit is used to acquire a target display result, wherein the target display result includes a display result obtained by displaying the n first images on a display device in the acquisition order of the n first images; A processing unit, configured to obtain a first relationship based on the position of the target object in the n first images, wherein the first relationship represents a relationship between the position of the target object in the images captured by the image acquisition device and time; The processing unit is used to obtain a second relationship based on the position of the target object in the target display result, wherein the second relationship represents a relationship between the position of the target object in the image displayed by the display device and time; The processing unit is configured to obtain the display delay of the display device based on a time difference between the first relationship and the second relationship.

8. An electronic device, characterized in that: include: A processor and a memory, wherein the memory is used to store computer program codes, wherein the computer program codes include computer instructions, and when the processor executes the computer instructions, the electronic device executes the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions. When the program instructions are executed by a processor, the processor is enabled to execute the method according to any one of claims 1 to 6.

10. A computer program product, characterized in that The computer program product comprises a computer program; when the computer program is run on a computer, the computer is caused to execute the method according to any one of claims 1 to 6.

Images