A force feedback method, device and electronic equipment for ultrasonic examination
Through ultrasonic image processing and control vector simulation of the probe's stress, real-time force feedback is provided, and hardware dependence and adaptability problems in remote ultrasonic examination systems are solved, improving the accuracy and safety of doctors' manipulation.
Patent Information
- Application Number
- CN202111045527.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-09-07
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2041-09-07
AI Technical Summary
In existing remote ultrasound examination systems, pressure data generation is related to specific hardware, it is difficult to adapt to different ultrasound devices, and the lack of intuitive force feedback makes it difficult for doctors to accurately grasp the pressure of the probe when manipulating the device, which may cause misjudgment and patients. harm.
By acquiring ultrasound images, we judge the contact state between the probe and the detection object, determine the motion state and depth of the probe, calculate the feedback force based on this information, and feed it back to the control terminal, and use digital image processing and adjustable analog feedback algorithm to realize force feedback , independent of pressure sensor hardware, adapted to different ultrasonic devices.
It realizes flexible application on different ultrasound devices, provides real-time force feedback when doctors operate the instrument, improves operation accuracy and safety, and meets functions that traditional hardware force feedback do not have, such as amplification of feedback force and contact. Pre-reminder.
Smart Images

Figure CN114767154B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of computer technology, and in particular to a force feedback method, device and electronic equipment for ultrasonic inspection. Background Art
[0002] The commonly used method for ultrasound examination at present is that the doctor is in close contact with the person being examined, and uses a handheld probe to press the body to obtain a suitable ultrasound image. With the development of medical device technology, it is becoming more and more common to use intelligent medical equipment in medical examinations and surgical operations. At the same time, due to the outbreak of the new crown epidemic, telemedicine has developed rapidly, and technologies such as remote diagnosis and remote treatment have become increasingly valued. Some remote ultrasound examination systems have been proposed.
[0003] The existing remote ultrasound examination system does not require doctors to have close contact with the patient during operation. Instead, they can perform examinations with the assistance of multi-angle cameras by remotely controlling medical equipment. This telemedicine system greatly expands the spatial practicality of ultrasound examinations. Doctors can examine patients from different departments of the same hospital, and even examine patients in different cities. This is very beneficial for epidemic prevention and control and the spatial allocation of medical resources.
[0004] However, in the existing remote ultrasound system, the pressure data is generated by collecting data from pressure sensors. This method is associated with a specific brand of hardware and system, and the data transmission and reception are subject to specified protocols. It is difficult to adapt to different ultrasound equipment and cannot be flexibly applied to different ultrasound platforms. At the same time, although pressure sensors are set up in the existing ultrasound system to collect the pressure data of the ultrasound probe on the human skin tissue, the pressure data is usually presented on the system screen in the form of numbers, etc., and is not directly reflected to the doctor, ignoring the force feedback required by the doctor when operating the instrument. The doctor can only judge the force on the probe through the pressure value collected by the pressure sensor, which is very unintuitive and inaccurate, and is not conducive to the doctor's accurate grasp of the pressure of the probe, which is easy to cause misjudgment and harm to the patient. Summary of the invention
[0005] To solve the problems in the related art, the embodiments of the present disclosure provide a virtual force feedback method, device and electronic device for ultrasonic inspection.
[0006] In a first aspect, an embodiment of the present disclosure provides a force feedback method for ultrasonic inspection.
[0007] Specifically, the force feedback method for ultrasonic inspection includes:
[0008] Acquiring ultrasound images;
[0009] determining the contact state between the ultrasonic probe and the detection object based on the ultrasonic image;
[0010] Determining the motion state of the ultrasonic probe when it just contacts the detection object, the real-time motion state of the ultrasonic probe, and the motion depth of the ultrasonic probe;
[0011] Determining the feedback force based on the motion state of the ultrasonic probe when it just contacts the detection object, the real-time motion state, and the motion depth;
[0012] The feedback force is fed back to the control terminal.
[0013] In combination with the first aspect, in a first implementation of the first aspect of the present disclosure, determining the contact state between the ultrasound probe and the detection object based on the ultrasound image includes:
[0014] Performing binarization image processing on the ultrasound image;
[0015] Calculating a ratio P of the number of 1 values to the number of 0 values in the ultrasound image after the binary image processing;
[0016] When the ratio P is greater than the first threshold When the ultrasonic probe is in contact with the detection object, it is determined that the ultrasonic probe is in contact with the detection object.
[0017] In combination with the first aspect, in a second implementation of the first aspect of the present disclosure, determining the motion state of the ultrasound probe when it just contacts the detection object and the real-time motion state of the ultrasound probe includes:
[0018] Based on the control vector transmitted by the controller twice when the ultrasonic probe just contacts the detection object, and Determine the motion state of the ultrasonic probe when it just contacts the detection object
[0019] Based on the real-time motion of the ultrasound probe, the controller samples the control vector twice. and Determine the real-time motion state of the ultrasound probe
[0020] In combination with the second implementation of the first aspect, in a third implementation of the first aspect of the present disclosure, the control vectors transmitted by the two samples are and as well as and are adjacent control vectors respectively.
[0021] In combination with the first implementation of the first aspect, in a fourth implementation of the first aspect of the present disclosure, determining the movement depth of the ultrasound probe includes:
[0022] Based on the motion state of the ultrasonic probe when it just contacts the detection object Real-time motion status of the ultrasonic probe And the control vector transmitted by the controller twice based on the real-time movement of the ultrasonic probe and The movement depth d of the ultrasound probe is determined.
[0023] In combination with the first implementation of the first aspect, in a fifth implementation of the first aspect of the present disclosure, determining the feedback force based on the motion state of the ultrasonic probe when it just contacts the detection object, the real-time motion state, and the motion depth includes:
[0024] Based on the real-time motion state of the ultrasonic probe The feedback force is determined by the movement depth d of the ultrasound probe.
[0025] In combination with the fifth implementation of the first aspect, in a sixth implementation of the first aspect of the present disclosure, the real-time motion state of the ultrasound probe is and the movement depth d of the ultrasonic probe to determine the feedback force includes:
[0026] Based on the real-time motion state of the ultrasonic probe The feedback force is determined by the product of the movement depth d of the ultrasound probe and the adjustable resistance coefficient k.
[0027] In a second aspect, an embodiment of the present disclosure provides a force feedback device for ultrasonic inspection.
[0028] Specifically, the force feedback device for ultrasonic inspection includes:
[0029] An acquisition module, configured to acquire an ultrasound image;
[0030] a judgment module, configured to judge the contact state between the ultrasound probe and the detection object based on the ultrasound image;
[0031] a determination module configured to determine a motion state of the ultrasonic probe when it just contacts the detection object, a real-time motion state of the ultrasonic probe, and a motion depth of the ultrasonic probe;
[0032] a calculation module configured to determine the feedback force based on the motion state of the ultrasound probe when it just contacts the detection object, the real-time motion state, and the motion depth;
[0033] The feedback module is configured to feed back the feedback force to the control terminal.
[0034] In a third aspect, an embodiment of the present disclosure provides an electronic device, comprising a memory and a processor, wherein the memory is used to store one or more computer instructions, and wherein the one or more computer instructions are executed by the processor to implement a method as described in any one of the first aspect and the first to sixth implementations of the first aspect.
[0035] In a fourth aspect, a computer-readable storage medium is provided in an embodiment of the present disclosure, on which computer instructions are stored. When the computer instructions are executed by a processor, a method as described in any one of the first aspect and the first to sixth implementations of the first aspect is implemented.
[0036] According to the technical solution provided in the embodiment of the present disclosure, the actual force condition of the ultrasound probe is simulated using the ultrasound image and the control vector of the controller. The feedback force data of the force condition is implemented based on a digital image processing algorithm and an adjustable analog feedback force algorithm. It does not rely on specific hardware such as pressure sensors, and can be easily run on different ultrasound devices and migrated to different ultrasound systems. It has strong flexibility.
[0037] At the same time, according to the technical solution provided by the embodiment of the present disclosure, the generated feedback force data is reflected on the control terminal of the doctor's control device. According to the contact between the ultrasound probe and the detection object, real-time force feedback with size and direction characteristics is added to the controller when the doctor controls the device, which helps the doctor to grasp the pressure of the device on the detection object, judge the position of the device and the detection object, and obtain better ultrasound images. In addition, adjustable parameters are set in the simulation feedback force algorithm. By setting these parameters, functions that traditional hardware force feedback does not have can be realized, such as amplifying feedback force, pre-alert of contact with the human body, etc., which can better meet the needs of doctors.
[0038] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] Other features, objectives and advantages of the present disclosure will become more apparent through the following detailed description of non-limiting embodiments in conjunction with the accompanying drawings. In the accompanying drawings:
[0040] Figure 1 A block diagram showing a force feedback method for ultrasonic inspection according to an embodiment of the present disclosure is shown;
[0041] Figure 2 A flow chart showing a force feedback method for ultrasonic inspection according to an embodiment of the present disclosure is shown;
[0042] Figure 3 A block diagram showing a method for determining a contact state between an ultrasonic probe and a detection object based on the ultrasonic image according to an embodiment of the present disclosure is shown;
[0043] Figure 4 A flow chart showing a method for determining the contact state between the ultrasonic probe and the detection object based on the ultrasonic image according to an embodiment of the present disclosure is shown;
[0044] Figure 5 A schematic diagram showing the motion state and the motion depth;
[0045] Figure 6 A block diagram of a force feedback device for ultrasonic inspection according to an embodiment of the present disclosure is shown;
[0046] Figure 7 An inspection platform using a force feedback method for ultrasonic inspection according to an embodiment of the present disclosure is shown;
[0047] Figure 8 A block diagram of an electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION
[0048] Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement them. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is obvious that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.
[0049] In the present disclosure, it should be understood that terms such as "includes," "comprising," or "having" are intended to indicate the presence of features, numbers, steps, behaviors, components, parts, or a combination thereof disclosed in the present specification, and are not intended to exclude the possibility that one or more other features, numbers, steps, behaviors, components, parts, or a combination thereof exist or are added.
[0050] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification, and should not be interpreted in an idealized or overly rigid manner.
[0051] In the case of using expressions such as "at least one of A, B, and C, etc.", it should generally be interpreted in accordance with the meaning of the expression generally understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but not be limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.). In the case of using expressions such as "at least one of A, B, or C, etc.", it should generally be interpreted in accordance with the meaning of the expression generally understood by those skilled in the art (for example, "a system having at least one of A, B, or C" should include but not be limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.). Those skilled in the art should also understand that any transitional conjunctions and / or phrases that substantially indicate two or more optional items, whether in the specification, claims, or drawings, should be understood to give the possibility of including one of these items, either of these items, or both of these items. For example, the phrase "A or B" should be understood to include the possibilities of "A" or "B", or "A and B".
[0052] It should also be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present disclosure may be combined with each other. The present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.
[0053] As mentioned above, in existing ultrasound systems, pressure data is generated by collecting data from pressure sensors. This method is associated with specific brands of hardware and systems, and the data transmission and reception are subject to specified protocols. It is difficult to adapt to different ultrasound equipment and cannot be flexibly applied to different ultrasound platforms. At the same time, although pressure sensors are set up in existing ultrasound systems to collect pressure data on the ultrasound probe on human skin tissue, the pressure data is usually presented on the system screen in digital form, etc., and is not directly reflected to the doctor, ignoring the force feedback required by the doctor when operating the instrument. The doctor can only judge the force on the probe through the pressure value collected by the pressure sensor, which is very unintuitive and inaccurate, and is not conducive to the doctor's accurate grasp of the pressure of the probe, which is easy to cause misjudgment and harm to the patient.
[0054] In the technical solution provided by the embodiment of the present disclosure, the real force condition of the ultrasonic probe is simulated by using the control vector of the ultrasonic image and the controller. The feedback force data of the force condition is realized based on the digital image processing algorithm and the adjustable simulation feedback force algorithm, which is independent of specific hardware such as the pressure sensor, and can be easily run on different ultrasonic devices and migrated to different ultrasonic systems, and has strong flexibility. At the same time, according to the technical solution provided by the embodiment of the present disclosure, the generated feedback force data is reflected on the control terminal of the doctor's control of the instrument. According to the contact condition between the ultrasonic probe and the detection object, the controller is added with real-time force feedback with size and direction characteristics when the doctor controls the instrument, which helps the doctor to grasp the pressure of the instrument on the detection object, judge the position of the instrument and the detection object, and obtain a better ultrasonic image. In addition, adjustable parameters are set in the simulation feedback force algorithm. By setting these parameters, functions that traditional hardware force feedback does not have, such as amplifying feedback force, pre-alert of contact with the human body, etc., can be realized, which can better meet the needs of doctors.
[0055] The embodiments of the present invention are further described in detail below with reference to the accompanying drawings.
[0056] Figure 1 A block diagram of a force feedback method for ultrasonic inspection according to an embodiment of the present disclosure is shown. Figure 2 A flow chart showing a force feedback method for ultrasound examination according to an embodiment of the present disclosure is shown.
[0057] like Figure 1 and 2 As shown, the method includes steps S101-S105.
[0058] In step S101 , an ultrasound image is acquired.
[0059] In step S102, the contact state between the ultrasound probe and the detection object is determined based on the ultrasound image.
[0060] In an optional implementation of this embodiment, Figure 3 and 4 As shown, in step S102, the contact state between the ultrasound probe and the detection object is determined based on the ultrasound image, including steps S1021-S1023.
[0061] In step S1021, the ultrasound image is subjected to binarization processing.
[0062] The binarization image processing of the ultrasound image may include binarizing the real-time ultrasound image into 1 and 0. Specifically, the pixel grayscale of each pixel in the ultrasound image may be determined pixel by pixel, and if the pixel grayscale is greater than or equal to a first threshold a, the pixel is represented by 1, and if the pixel grayscale is less than the first threshold a, the pixel is represented by 0. The value of the first threshold a is determined according to the imaging effect of different color ultrasound equipment, and the preferred value range of a proposed in this method is 150-200.
[0063] In step S1022, a ratio P of the number of 1 values to the number of 0 values in the ultrasound image after the binary image processing is calculated.
[0064] Specifically, the pixel value of 1 in the ultrasound image is counted as n1, and the pixel value of 0 in the ultrasound image is counted as n0, then
[0065] In step S1023, when the ratio P is greater than the second threshold When the ultrasonic probe is in contact with the detection object, it is determined that the ultrasonic probe is in contact with the detection object.
[0066] Specifically, compare the ratio P with the second threshold when When the ultrasonic probe is in contact with the detection object, It is determined that the ultrasonic probe is not in contact with the detection object.
[0067] Among them, the second threshold is an adjustable variable, representing the sensitivity of judging whether the probe contacts the detection object. The larger its value, the more sensitive the system is in judging whether the probe contacts the detection object, and vice versa. The adjustment can be made according to the imaging effect of the color Doppler ultrasound equipment and the specific conditions of the detection object, which can be a person or an animal. The specific conditions of the detection object can include the category of the detection object, the gender of the detection object, the fatness or thinness of the detection object, etc. The optimal value range is 1.8-2.3.
[0068] In step S103, the motion state of the ultrasonic probe when it just contacts the detection object, the real-time motion state of the ultrasonic probe, and the motion depth of the ultrasonic probe are determined.
[0069] In an optional implementation of this embodiment, determining the motion state of the ultrasonic probe when it just contacts the detection object may include, based on the control vector transmitted by the controller twice when the ultrasonic probe just contacts the detection object. and Determine the motion state of the ultrasonic probe when it just contacts the detection object The state when the ultrasonic probe just contacts the detection object is determined by the above step S102.
[0070] In the equipment for ultrasound examination, the ultrasound probe can be moved under the clamp of the robotic arm. The doctor controls the controller to issue various commands such as forward and backward. The controller converts the doctor's operation into a control vector and sends it to the robotic arm. After the robotic arm analyzes the control vector, it clamps the ultrasound probe and makes different movements, thereby realizing the doctor's control of the ultrasound probe. During the control process, the controller continuously sends control vectors to the robotic arm. When it is determined that the ultrasound probe has changed from a non-contact detection object to a contact state, the control vector is sampled twice to obtain the control vector transmitted by the controller twice. and The two samplings may be two non-continuous samplings or two continuous samplings. When the two samplings are two continuous samplings, the control vector and The sampling frequency is affected by network conditions and can be set accordingly. In this method, the sampling frequency is proposed to be 1000-3000Hz.
[0071] Specifically, the formula The motion state of the ultrasonic probe when it just contacts the detection object is calculated.
[0072] In another optional implementation of this embodiment, determining the real-time motion state of the ultrasound probe may include, based on the control vector transmitted by the controller twice when the ultrasound probe performs real-time detection on the detection object. and Determine the real-time motion state of the ultrasound probe
[0073] In the equipment for ultrasound examination, the ultrasound probe can be moved under the clamp of the robotic arm. The doctor controls the controller to issue various commands such as forward and backward. The controller converts the doctor's operation into a control vector and sends it to the robotic arm. After the robotic arm analyzes the control vector, it clamps the ultrasound probe and makes different movements, thereby realizing the doctor's control of the ultrasound probe. During the control process, the controller continuously sends control vectors to the robotic arm. When the ultrasound probe has contacted the detection object and is in a real-time detection state of the detection object, the control vector is sampled twice to obtain the control vector transmitted by the controller twice. and The two samplings may be two consecutive samplings, in which case the control vector and The sampling frequency is affected by network conditions and can be set accordingly. In this method, the sampling frequency is proposed to be 1000-3000Hz.
[0074] Specifically, the formula The motion state of the ultrasonic probe when it just contacts the detection object is calculated.
[0075] In another optional implementation of this embodiment, determining the movement depth of the ultrasound probe may include determining the movement depth of the ultrasound probe based on the movement state of the ultrasound probe when the ultrasound probe just contacts the detection object. Real-time motion status of the ultrasonic probe And the control vector transmitted by the controller twice based on the real-time movement of the ultrasonic probe and Determine the movement depth d of the ultrasonic probe. Specifically, the formula The movement depth of the ultrasound probe is calculated.
[0076] In step S104, the feedback force is determined based on the motion state of the ultrasound probe when it first contacts the detection object, the real-time motion state, and the motion depth.
[0077] In an optional implementation of this embodiment, according to the formula The feedback force is calculated, wherein k is an adjustable resistance coefficient, which is set according to the specific conditions of the detected object. In this method, k=60-80N / cm is proposed.
[0078] From the above records, we can know that the motion vector and They represent the motion state of the ultrasonic probe just touching the detection object and the real-time motion state, respectively, and are both calculated based on the vector difference of the control vector. The vector difference represents the direction and distance of the ultrasonic probe movement. The direction of the ultrasonic probe movement is represented by the motion vector of the ultrasonic probe. The angle between the probe and the skin surface is represented by the motion vector value, and the distance of the probe movement is represented by the motion vector value. Figure 5 , combined with the above calculation formula, we can know that the motion vector of the ultrasound probe is The larger the angle with the skin surface, the greater the movement depth, and thus the greater the resistance, i.e. the feedback force f; the movement vector of the ultrasound probe The smaller the angle with the skin surface, the smaller the movement depth will be, and the smaller the resistance, i.e., the feedback force f, will be. When the angle remains unchanged, the larger the movement vector value, the larger the movement depth will be, and the larger the resistance, i.e., the feedback force f, will be; the smaller the movement vector value, the smaller the movement depth will be, and the smaller the resistance, i.e., the feedback force f, will be, and the direction of the feedback force f is always opposite to the real-time movement direction of the probe, so it can well simulate the basic characteristics of real force feedback.
[0079] In step S105, the feedback force is fed back to the controller terminal.
[0080] In an optional implementation of this embodiment, after obtaining the feedback force f, the feedback force signal that can characterize the real-time force added to the controller when the doctor manipulates the instrument is fed back to the controller terminal, so that the doctor can sense the size and direction of the ultrasound probe pressure through the feel of the controller and obtain a force feedback feeling similar to that in traditional ultrasound examinations.
[0081] According to the technical solution provided in the embodiment of the present disclosure, the actual force condition of the ultrasound probe is simulated by utilizing the ultrasound image and the control vector of the controller. The feedback force data of the force condition is realized based on a digital image processing algorithm and an adjustable simulation feedback force algorithm. It does not rely on specific hardware such as pressure sensors, and can be easily run on different ultrasound devices and migrated to different ultrasound systems. It has strong flexibility.
[0082] At the same time, according to the technical solution provided by the embodiment of the present disclosure, the generated feedback force data is reflected on the control terminal of the doctor's control device. According to the contact between the ultrasound probe and the detection object, real-time force feedback with size and direction characteristics is added to the controller when the doctor controls the device, which helps the doctor to grasp the pressure of the device on the detection object, judge the position of the device and the detection object, and obtain better ultrasound images. In addition, adjustable parameters are set in the simulation feedback force algorithm. By setting these parameters, functions that traditional hardware force feedback does not have can be realized, such as amplifying feedback force, pre-alert of contact with the human body, etc., which can better meet the needs of doctors.
[0083] Figure 6 A block diagram of a force feedback device for ultrasonic inspection according to an embodiment of the present disclosure is shown, wherein the device can be implemented as part or all of an electronic device through software, hardware or a combination of both.
[0084] like Figure 6 As shown, the force feedback device for ultrasonic inspection includes an acquisition module 601, a judgment module 602, a determination module 603, a calculation module 604 and a feedback module 605.
[0085] An acquisition module 601 is configured to acquire an ultrasound image;
[0086] A determination module 602 is configured to determine a contact state between the ultrasound probe and the detection object based on the ultrasound image;
[0087] A determination module 603 is configured to determine the motion state of the ultrasonic probe when it just contacts the detection object, the real-time motion state of the ultrasonic probe, and the motion depth of the ultrasonic probe;
[0088] A calculation module 604 is configured to determine the feedback force based on the motion state of the ultrasound probe when it just contacts the detection object, the real-time motion state, and the motion depth;
[0089] The feedback module 605 is configured to feed back the feedback force to the control terminal.
[0090] According to an embodiment of the present disclosure, determining the contact state between the ultrasound probe and the detection object based on the ultrasound image includes:
[0091] Performing binarization image processing on the ultrasound image;
[0092] Calculating a ratio P of the number of 1 values to the number of 0 values in the ultrasound image after the binary image processing;
[0093] When the ratio P is greater than the first threshold When the ultrasonic probe is in contact with the detection object, it is determined that the ultrasonic probe is in contact with the detection object.
[0094] According to an embodiment of the present disclosure, determining the motion state of the ultrasound probe when it just contacts the detection object and the real-time motion state of the ultrasound probe includes:
[0095] Based on the control vector transmitted by the controller twice when the ultrasonic probe just contacts the detection object, and Determine the motion state of the ultrasonic probe when it just contacts the detection object
[0096] Based on the real-time motion of the ultrasound probe, the controller samples the control vector twice. and Determine the real-time motion state of the ultrasound probe
[0097] According to an embodiment of the present disclosure, the control vector transmitted by the two samplings is and as well as and are adjacent control vectors respectively.
[0098] According to an embodiment of the present disclosure, determining the movement depth of the ultrasonic probe includes:
[0099] Based on the motion state of the ultrasonic probe when it just contacts the detection object Real-time motion status of the ultrasonic probe And the control vector transmitted by the controller twice based on the real-time movement of the ultrasonic probe and The movement depth d of the ultrasound probe is determined.
[0100] According to an embodiment of the present disclosure, the step of determining the feedback force based on the motion state of the ultrasonic probe when it just contacts the detection object, the real-time motion state, and the motion depth includes:
[0101] Based on the real-time motion state of the ultrasonic probe The feedback force is determined by the movement depth d of the ultrasound probe.
[0102] According to an embodiment of the present disclosure, the real-time motion state of the ultrasonic probe and the movement depth d of the ultrasonic probe to determine the feedback force includes:
[0103] Based on the real-time motion state of the ultrasonic probe The feedback force is determined by the product of the movement depth d of the ultrasound probe and the adjustable resistance coefficient k.
[0104] According to the technical solution provided in the embodiment of the present disclosure, the actual force condition of the ultrasound probe is simulated by utilizing the ultrasound image and the control vector of the controller. The feedback force data of the force condition is realized based on a digital image processing algorithm and an adjustable simulation feedback force algorithm. It does not rely on specific hardware such as pressure sensors, and can be easily run on different ultrasound devices and migrated to different ultrasound systems. It has strong flexibility.
[0105] At the same time, according to the technical solution provided by the embodiment of the present disclosure, the generated feedback force data is reflected on the control terminal of the doctor's control device. According to the contact between the ultrasound probe and the detection object, real-time force feedback with size and direction characteristics is added to the controller when the doctor controls the device, which helps the doctor to grasp the pressure of the device on the detection object, judge the position of the device and the detection object, and obtain better ultrasound images. In addition, adjustable parameters are set in the simulation feedback force algorithm. By setting these parameters, functions that traditional hardware force feedback does not have can be realized, such as amplifying feedback force, pre-alert of contact with the human body, etc., which can better meet the needs of doctors.
[0106] Figure 7 An inspection platform using the force feedback method for ultrasonic inspection according to an embodiment of the present disclosure is shown.
[0107] like Figure 7The inspection platform includes a control end and an inspection end. The control end includes a controller controlled by a doctor and a controller terminal for realizing signal processing. The doctor can send a control signal to the controller terminal through the controller, and the controller terminal transmits the control signal to the inspection end. The controller terminal can also receive the ultrasound image returned by the inspection end, and calculate the feedback force based on the ultrasound image and the control vector in the control signal, and finally feed back the feedback force to the controller controlled by the doctor. The inspection end includes a mechanical arm and an ultrasound device. The ultrasound device includes an ultrasound probe, and the mechanical arm is used to clamp the ultrasound probe. The inspection end can also include a mechanical arm terminal for receiving the control signal transmitted by the controller terminal, controlling the mechanical arm movement based on the control signal to obtain an ultrasound image, and returning the ultrasound image to the controller terminal. The mechanical arm terminal can directly return the original ultrasound image collected, or return an image after binarization of the collected image.
[0108] During an ultrasound examination, the doctor controls the robotic arm to hold the probe and scan the human body. The control command is issued by the controller, transmitted to the controller terminal, and then transmitted to the robotic arm terminal through the network link. After receiving the control command, the robotic arm terminal decodes it and sends a control signal to the robotic arm to complete the control of the robotic arm's movement.
[0109] Those skilled in the art will appreciate that the control end and the inspection end may be located in different hospitals, or in different departments or rooms of the same hospital, or in the same room, and the present disclosure does not limit this. The doctor sends a control signal to the robotic arm terminal of the inspection end through the controller terminal of the control end, and obtains the ultrasonic image of the inspection end from the control end.
[0110] In the above process, the ultrasound device will continuously process the ultrasound signal generated by the probe and generate real-time ultrasound images. The ultrasound device is connected to the robotic arm terminal via a hardware link, and the real-time ultrasound image is continuously transmitted to the robotic arm terminal. The binary image processing algorithm carried by it determines whether the probe is in contact with the human body, and transmits the information back to the controller terminal via the network link. After the controller terminal receives the signal that the probe is in contact with the human body, the feedback force generation algorithm carried by it starts to work. It retains the motion information sent by the controller at this time, and continues to receive the control signal of the controller thereafter, generates simulated feedback force, and feeds it back to the controller. In this way, the doctor can judge the movement of the probe based on the real-time resistance transmitted by the remote control.
[0111] The present disclosure also discloses an electronic device, Figure 8 A block diagram of an electronic device according to an embodiment of the present disclosure is shown.
[0112] like Figure 8As shown, the electronic device 800 includes a memory 801 and a processor 802, wherein the memory 801 is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor 802 to implement the following operations:
[0113] Acquiring ultrasound images;
[0114] determining the contact state between the ultrasonic probe and the detection object based on the ultrasonic image;
[0115] Determining the motion state of the ultrasonic probe when it just contacts the detection object, the real-time motion state of the ultrasonic probe, and the motion depth of the ultrasonic probe;
[0116] Determining the feedback force based on the motion state of the ultrasonic probe when it just contacts the detection object, the real-time motion state, and the motion depth;
[0117] The feedback force is fed back to the control terminal.
[0118] According to an embodiment of the present disclosure, determining the contact state between the ultrasound probe and the detection object based on the ultrasound image includes:
[0119] Performing binarization image processing on the ultrasound image;
[0120] Calculating a ratio P of the number of 1 values to the number of 0 values in the ultrasound image after the binary image processing;
[0121] When the ratio P is greater than the first threshold When the ultrasonic probe is in contact with the detection object, it is determined that the ultrasonic probe is in contact with the detection object.
[0122] According to an embodiment of the present disclosure, determining the motion state of the ultrasound probe when it just contacts the detection object and the real-time motion state of the ultrasound probe includes:
[0123] Based on the control vector transmitted by the controller twice when the ultrasonic probe just contacts the detection object, and Determine the motion state of the ultrasonic probe when it just contacts the detection object
[0124] Based on the real-time motion of the ultrasound probe, the controller samples the control vector twice. and Determine the real-time motion state of the ultrasound probe
[0125] According to an embodiment of the present disclosure, the control vector transmitted by the two samplings is and as well as and are adjacent control vectors respectively.
[0126] According to an embodiment of the present disclosure, determining the movement depth of the ultrasonic probe includes:
[0127] Based on the motion state of the ultrasonic probe when it just contacts the detection object Real-time motion status of the ultrasonic probe And the control vector transmitted by the controller twice based on the real-time movement of the ultrasonic probe and The movement depth d of the ultrasound probe is determined.
[0128] According to an embodiment of the present disclosure, determining the feedback force based on the motion state of the ultrasonic probe when it just contacts the detection object, the real-time motion state, and the motion depth includes:
[0129] Based on the real-time motion state of the ultrasonic probe The feedback force is determined by the movement depth d of the ultrasound probe.
[0130] According to an embodiment of the present disclosure, the real-time motion state of the ultrasonic probe and the movement depth d of the ultrasonic probe to determine the feedback force includes:
[0131] Based on the real-time motion state of the ultrasonic probe The feedback force is determined by the product of the movement depth d of the ultrasound probe and the adjustable resistance coefficient k.
[0132] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0133] The units or modules involved in the embodiments described in the present disclosure may be implemented by software or programmable hardware. The units or modules described may also be set in a processor, and the names of these units or modules do not constitute limitations on the units or modules themselves in some cases.
[0134] As another aspect, the present disclosure further provides a computer-readable storage medium, which may be a computer-readable storage medium included in the electronic device or computer system in the above embodiment; or a computer-readable storage medium that exists independently and is not assembled into a device. The computer-readable storage medium stores one or more programs, and the programs are used by one or more processors to execute the method described in the present disclosure.
[0135] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present disclosure is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the inventive concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other.
Claims
1. A force feedback method for ultrasonic inspection, comprising: Acquiring ultrasound images; determining the contact state between the ultrasonic probe and the detection object based on the ultrasonic image; Determining the motion state of the ultrasonic probe when it just contacts the detection object, the real-time motion state of the ultrasonic probe, and the motion depth of the ultrasonic probe; Determining the feedback force based on the motion state of the ultrasonic probe when it just contacts the detection object, the real-time motion state, and the motion depth; Feeding back the feedback force to a controller terminal; Wherein, the determining of the motion state of the ultrasonic probe when it just contacts the detection object and the real-time motion state of the ultrasonic probe comprises: Based on the control vector transmitted by the controller twice when the ultrasonic probe just contacts the detection object, and Determine the motion state of the ultrasonic probe when it just contacts the detection object ; Based on the real-time motion of the ultrasound probe, the controller samples the control vector twice. and Determine the real-time motion state of the ultrasound probe ; Determining the movement depth of the ultrasonic probe comprises: Based on the motion state of the ultrasonic probe when it just contacts the detection object , the real-time motion state of the ultrasonic probe And the control vector transmitted by the controller twice based on the real-time movement of the ultrasonic probe and Determine the movement depth of the ultrasound probe ; in, .
2. The method according to claim 1, wherein: Determining the contact state between the ultrasonic probe and the detection object based on the ultrasonic image includes: Performing binarization image processing on the ultrasonic image; Calculating a ratio P of the number of 1 values to the number of 0 values in the ultrasound image after the binary image processing; When the ratio P is greater than the second threshold When the ultrasonic probe is in contact with the detection object, it is determined that the ultrasonic probe is in contact with the detection object.
3. The method according to claim 1, wherein: The control vector of the two sampling transfers and ,as well as and are adjacent control vectors respectively.
4. The method according to claim 1, wherein: The step of determining the feedback force based on the motion state of the ultrasonic probe when it just contacts the detection object, the real-time motion state, and the motion depth comprises: Based on the real-time motion state of the ultrasonic probe and the depth of ultrasound probe movement The feedback force is determined.
5. The method according to claim 4, wherein: The real-time motion state based on the ultrasonic probe and the depth of ultrasound probe movement Determining the feedback force includes: Based on the real-time motion state of the ultrasonic probe , the movement depth of the ultrasound probe and the adjustable drag coefficient determine the feedback force.
6. A force feedback device for ultrasonic examination, comprising: An acquisition module, configured to acquire an ultrasound image; a judgment module, configured to judge the contact state between the ultrasonic probe and the detection object based on the ultrasonic image; a determination module configured to determine a motion state of the ultrasonic probe when it just contacts the detection object, a real-time motion state of the ultrasonic probe, and a motion depth of the ultrasonic probe; a calculation module configured to determine the feedback force based on the motion state of the ultrasound probe when it just contacts the detection object, the real-time motion state, and the motion depth; A feedback module, configured to feed back the feedback force to a control terminal; Wherein, the determining of the motion state of the ultrasonic probe when it just contacts the detection object and the real-time motion state of the ultrasonic probe comprises: Based on the control vector transmitted by the controller twice when the ultrasonic probe just contacts the detection object, and Determine the motion state of the ultrasonic probe when it just contacts the detection object ; Based on the real-time motion of the ultrasound probe, the controller samples the control vector twice. and Determine the real-time motion state of the ultrasound probe ; Determining the movement depth of the ultrasonic probe comprises: Based on the motion state of the ultrasonic probe when it just contacts the detection object , the real-time motion state of the ultrasonic probe And the control vector transmitted by the controller twice based on the real-time movement of the ultrasonic probe and Determine the movement depth of the ultrasound probe ; in, .
7. An electronic device, characterized in that: The method comprises a memory and a processor; wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method steps described in any one of claims 1-5.
8. A readable storage medium having computer instructions stored thereon, characterized in that: When the computer instructions are executed by a processor, the method steps described in any one of claims 1 to 5 are implemented.
