Interaction method, device, electronic device and storage medium

Abstract

The present invention relates to an interactive method, device, electronic device and storage medium, wherein the method comprises: obtaining a real-time image including a target object; drawing a target primitive based on the target object so that the target primitive and the real-time image are displayed in a visual interface; receiving an adjustment instruction for the target primitive so that an adjustment execution device responds to the adjustment instruction and performs corresponding adjustment. The present invention optimizes the traditional workflow, improves the success rate of patient photography and reduces the complexity of doctor's operation by accurately adjusting the exposure range according to visual positioning.

Description

Technical Field

[0001] The present invention relates to the field of image processing and automation technology, and in particular to an interaction method, device, electronic equipment and storage medium. Background Art

[0002] Currently, medical technology is developing rapidly. The use of medical X-ray instruments can clearly reflect the patient's condition and facilitate doctors' diagnosis and treatment. However, the current mainstream medical X-ray instruments are not combined with cameras. Therefore, when performing X-ray examinations on patients, doctors are required to manually adjust the medical equipment and repeatedly confirm the patient's positioning posture, which greatly affects the shooting efficiency and patient health.

[0003] There are also some medical radiography instruments combined with cameras in the prior art, but some of them simply transmit the patient's position through the display without considering the details, so the shooting results are still not ideal; the other part displays the acquired video stream on the display, and adds the recognition of the human body position, the head rotation angle, the body tilt angle, and the left and right center distance, etc. However, these are just some prompt information and do not support the doctor's adjustment of details.

[0004] Therefore, how to more conveniently adjust medical X-ray instruments to display more accurate patient information and thus improve diagnostic efficiency has always been a problem that needs to be solved urgently. Summary of the invention

[0005] In view of this, it is necessary to provide an interactive method to solve the problem of how to conveniently adjust medical radiation equipment to improve diagnostic efficiency.

[0006] In order to solve the above problem, the present invention provides an interaction method, comprising:

[0007] Acquire a real-time image including a target object;

[0008] Draw a target graphic element based on the target object, so that the target graphic element and the real-time image are displayed in the visualization interface;

[0009] An adjustment instruction for a target graphic element is received, so that an adjustment execution device responds to the adjustment instruction and performs corresponding adjustment.

[0010] Optionally, draw the target primitive based on the target object, including:

[0011] Obtain depth information of real-time images and construct a three-dimensional space coordinate system based on the depth information;

[0012] Obtaining the first coordinate information of the target object in the three-dimensional space coordinate system;

[0013] Converting first coordinate information in the three-dimensional space coordinate system into second coordinate information in the two-dimensional space coordinate system based on a preset dimensional conversion relationship;

[0014] Draw the target primitive based on the second coordinate information.

[0015] Optionally, determining a preset dimension conversion relationship includes:

[0016] Using a preset calibration algorithm to establish the correlation between the three-dimensional space coordinate system and the two-dimensional space coordinate system;

[0017] The preset dimension conversion relationship is determined based on the association relationship.

[0018] Optionally, the target graphics element and the real-time image are displayed in a visualization interface, including:

[0019] The target graphic element is overlapped with the real-time image and displayed in the visualization interface, and the target graphic element is displayed on the real-time image.

[0020] Optionally, the real-time image also includes a fixed object and a moving object; displaying the target image element on the real-time image includes:

[0021] Identify the relative positions of fixed and moving objects;

[0022] If the overlapping area between the fixed object and the moving object reaches a preset overlapping threshold, the target graphic element is displayed on the surface of the moving object in the real-time image;

[0023] If the overlapping area between the fixed object and the moving object does not reach a preset overlapping threshold, the target graphic element is displayed on the surface of the fixed object in the real-time image.

[0024] Optionally, the target image element includes a light field range; receiving an adjustment instruction for the target image element so that the adjustment execution device responds to the adjustment instruction and performs corresponding adjustment, including:

[0025] A drag adjustment instruction for the light field range is received, so that the adjustment execution device responds to the light field range adjustment instruction to adjust the size of the light field range.

[0026] Optionally, the target graphic element further includes a splicing range and a movement range of the adjustment execution device; receiving an adjustment instruction for the target graphic element so that the adjustment execution device responds to the adjustment instruction and performs corresponding adjustment, including:

[0027] A drag adjustment instruction for the splicing range is received according to the movement range of the adjustment execution device, so that the adjustment execution device responds to the splicing range adjustment instruction to adjust the size of the splicing range.

[0028] The present invention also provides an interactive device, comprising:

[0029] An image acquisition module, used to acquire a real-time image including a target object;

[0030] A graphic element drawing module, used for drawing a target graphic element based on a target object, so that the target graphic element and the real-time image are displayed in a visualization interface;

[0031] The graphic element adjustment module is used to receive an adjustment instruction for a target graphic element, so that the adjustment execution device responds to the adjustment instruction and performs corresponding adjustment.

[0032] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned interaction method when executing the computer program.

[0033] The present invention also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of the above-mentioned interaction method are implemented.

[0034] When performing a radiographic examination on a patient, the present invention can preliminarily guide the patient's standing position by acquiring a real-time image, and then draw a target graphic element based on the target object, so that the target graphic element and the real-time image are displayed in a visual interface, and receive an adjustment instruction for the target graphic element, so that the adjustment execution device responds to the adjustment instruction and performs a corresponding adjustment, that is, the exposure range is accurately adjusted according to the visual positioning, and the adjustment execution device can automatically adjust according to the adjustment instruction, thereby avoiding the tedious manual adjustment, optimizing the traditional workflow, improving the success rate of patient shooting, and reducing the complexity of the doctor's operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 A flowchart of an interactive method provided by an embodiment of the present invention;

[0036] Figure 2 A schematic diagram of the structure of a medical ray instrument combined with a camera provided by an embodiment of the present invention;

[0037] Figure 3 A schematic diagram of a process of drawing a target graphic element based on a target object provided by an embodiment of the present invention;

[0038] Figure 4 A relative position relationship diagram of a human body surface and a housing surface relative to a radiation source or a camera provided in an embodiment of the present invention;

[0039] Figure 5 An effect diagram of a target graphic element and a real-time image being overlapped and displayed in a visualization interface provided by an embodiment of the present invention;

[0040] Figure 6 A schematic diagram of the structure of an interactive device provided by an embodiment of the present invention;

[0041] Figure 7 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0042] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not used to limit the scope of the present invention.

[0043] See also Figure 1 , Figure 1 A flowchart of an interactive method provided in an embodiment of the present invention includes:

[0044] Step S101: acquiring a real-time image including a target object;

[0045] Step S102: drawing a target graphic element based on the target object, so that the target graphic element and the real-time image are displayed in the visualization interface;

[0046] Step S103: receiving an adjustment instruction for the target graphic element, so that the adjustment execution device responds to the adjustment instruction and performs corresponding adjustment.

[0047] In some embodiments of the present invention, the execution subject of the above steps may be, but is not limited to, a medical terminal or a corresponding server, such as a physician operation terminal, a medical radiation instrument control device, etc. It is understandable that the physician operation terminal is not limited to a tablet, a PC, a handheld terminal, a portable terminal, a head-mounted terminal, etc.

[0048] As a specific embodiment, in step S101, the real-time image including the target object can be collected by a camera combined with a medical X-ray instrument. The doctor's operating terminal can communicate with the camera and obtain the real-time image using a wired network / wireless network. This acquisition method is a common technology and will not be described in detail here.

[0049] The target object may be a component in a medical radiation instrument system, specifically a detector, an ionization chamber, etc. By acquiring a real-time image including the target object, the positions of the components of the medical radiation instrument may be roughly displayed in a visual interface.

[0050] As a specific embodiment, in step S102, the target primitive is a semi-transparent basic graphic for more intuitively displaying the target object in the visualization interface, and the positional relationship between the ROI (Region Of Interest) and the patient's part to be examined can be intuitively reflected through the target primitive. Drawing the target object to obtain the target primitive can be understood as converting the components of the medical radiation instrument system in the real world into semi-transparent basic graphics in the image.

[0051] After the target primitive is drawn, the target primitive and the real-time image can be displayed in the visualization interface at the same time. It can be understood that the pixel information of the target primitive and the real-time image is cached in the physician operation terminal, which can be specifically stored in the cache of the graphics card. The display data in the graphics card cache is continuously refreshed on the screen, and the target primitive and the real-time image are displayed in the visualization interface of the physician operation terminal.

[0052] As a specific embodiment, in step S103, the adjustment instruction for the target element may be operations such as dragging, enlarging and reducing the target element by the physician; wherein the adjustment execution device is also a component of the medical radiation instrument, specifically a beam limiter, and the physician operation terminal is communicatively connected with the medical radiation instrument system, so when the adjustment operation is performed on the target element in the physician operation terminal, the beam limiter can automatically respond to the adjustment instruction and make corresponding adjustments.

[0053] It is understandable that in the traditional workflow, doctors are required to manually adjust the execution equipment and repeatedly confirm the patient's positioning posture, which not only delays the efficiency of diagnosis and treatment, but also has a certain impact on the patient's body.

[0054] When performing a radiographic examination on a patient, the present invention can preliminarily guide the patient's standing position by acquiring a real-time image, and then draw a target graphic element based on the target object, so that the target graphic element and the real-time image are displayed in a visual interface, and receive an adjustment instruction for the target graphic element, so that the adjustment execution device responds to the adjustment instruction and performs a corresponding adjustment, that is, the exposure range is accurately adjusted according to the visual positioning, and the adjustment execution device can automatically adjust according to the adjustment instruction, thereby avoiding the tedious manual adjustment, optimizing the traditional workflow, improving the success rate of patient shooting, and reducing the complexity of the doctor's operation.

[0055] In one embodiment of the present invention, in order not to affect the display of the light field, the camera combined with the medical radiation instrument can be installed on the side of the beam limiter in the medical radiation instrument. Figure 2 , Figure 2 The schematic diagram of the structure of a medical ray instrument combined with a camera and a physician operation terminal provided in an embodiment of the present invention. The image can be collected in real time by a camera installed on one side of the beam limiter. After the patient reaches the designated positioning position, the doctor can monitor the patient's position and status in real time through the visual interface of the terminal.

[0056] In one embodiment of the present invention, referring to Figure 3 , Figure 3 A flowchart of drawing target graphics based on target objects is provided, including:

[0057] Step S1021: acquiring depth information of the real-time image, and constructing a three-dimensional space coordinate system based on the depth information;

[0058] Step S1022: obtaining first coordinate information of the target object in the three-dimensional space coordinate system;

[0059] Step S1023: converting the first coordinate information in the three-dimensional space coordinate system into second coordinate information in the two-dimensional space coordinate system based on a preset dimensional conversion relationship;

[0060] Step S1024: Draw the target primitive based on the second coordinate information.

[0061] It should be noted that the various components of the medical radiation instrument include a detector and an automatic exposure control ionization chamber. During the radiation examination process, the target objects include the detector imaging range, the automatic exposure control ionization chamber and the light field range and stitching range of the radiation radiation emitted by the beam limiter. Among them, the detector imaging range can be used to observe whether the patient's part to be inspected is located in the detector, and the patient's position can be roughly adjusted; the automatic exposure control ionization chamber can be used to observe the selection of the measurement area, which is convenient for locating the part to be inspected; the light field range can be used to preview the imaging area after the film is formed; and the stitching range can be used to observe more information about the part to be inspected.

[0062] However, when the patient reaches the designated position, the target object will inevitably be blocked, so it is necessary to draw the primitives of the target object.

[0063] As a specific embodiment, in step S1021, the depth information represents the distance information between the camera and the target object. The three-dimensional space coordinate system includes the world coordinate system, which is a three-dimensional rectangular coordinate system. It mainly describes the spatial position of the camera and the object to be measured, that is, the target object in the real world. Specifically, after obtaining the depth information of the real-time image, the three-dimensional space coordinate system can be established with reference to the position of the medical X-ray instrument and the camera.

[0064] As a specific embodiment, in step S1022, after the three-dimensional space coordinate system is established, first coordinate information of the target object in the three-dimensional space coordinate system may be acquired according to the depth information of the target object.

[0065] As a specific embodiment, in step S1023, the preset dimensional conversion relationship includes a conversion relationship between a three-dimensional space coordinate system and a two-dimensional space coordinate system.

[0066] Specifically, determining the preset dimension conversion relationship includes:

[0067] The relationship between the three-dimensional space coordinate system and the two-dimensional space coordinate system established by using a preset calibration algorithm;

[0068] The preset dimension conversion relationship is determined based on the association relationship.

[0069] Among them, the preset calibration algorithm can be a camera calibration algorithm. The specific process can be a process of constructing a mathematical model to approximate camera imaging, which realizes the mutual conversion of three-dimensional world coordinates and two-dimensional pixel coordinates. The purpose of camera calibration is actually to find out the relevant parameters of this model, among which the necessary parameters are the camera's intrinsic parameter matrix and the camera's extrinsic parameter matrix. By solving the relevant parameters, the three-dimensional world coordinates and the two-dimensional pixel coordinates can be freely converted. Exemplarily, Zhang Zhengyou's camera calibration method can be used. The specific process of using Zhang Zhengyou's camera calibration method to solve the relevant parameters of the above model will not be repeated here.

[0070] After obtaining the preset dimension conversion relationship, the first coordinate information in the three-dimensional space coordinate system can be converted into the second coordinate information in the two-dimensional space coordinate system according to the dimension conversion relationship. The first coordinate information includes the position information obtained according to the image depth information in the custom world coordinate system, and the second coordinate information includes the pixel information of the pixel point in the pixel coordinate system.

[0071] As a specific embodiment, in step S1024, the target primitive is a semi-transparent basic graphic for displaying the target object more intuitively in the visualization interface. After obtaining the second coordinate information, that is, the information of the pixel points of each component of the medical ray instrument in the pixel coordinate system, the primitive can be drawn by changing the transparency of the pixel points.

[0072] Specifically, the coordinate data returned by the camera, that is, the coordinate information of the world coordinate system, can be converted into pixel coordinates by coordinate dimension conversion, and then the image element of the detector imaging area can be drawn according to the pixel coordinates, while the relative position of the automatic exposure control ionization chamber and the detector imaging area is fixed. Therefore, drawing can be done more quickly, and by drawing the target object into the target image element, the positional relationship between the patient's part to be examined and the set area of ​​interest can be intuitively reflected in the visual interface, which is convenient for doctors to make a diagnosis.

[0073] In one embodiment of the present invention, displaying the target graphic element and the real-time image in a visualization interface includes:

[0074] The target graphic element is overlapped with the real-time image and displayed in the visualization interface, and the target graphic element is displayed on the real-time image.

[0075] Among them, overlapping and displaying the target primitive and the real-time image in the visualization interface can be understood as the overlapping display of the semi-transparent basic graphic represented by the target primitive and the entity of the medical radiation instrument component represented by the target object. Specifically, the layers of the target primitive and the real-time image can be set to make the two overlap and display. More specifically, in order to facilitate the observation of the relative position relationship between each target primitive and the part to be inspected of the patient, the image of the real-time image can be set at the bottom layer of the image.

[0076] In one embodiment of the present invention, the real-time image also includes a fixed object and a moving object; displaying the target image element on the real-time image includes:

[0077] Identify the relative positions of fixed and moving objects;

[0078] If the overlapping area between the fixed object and the moving object reaches a preset overlapping threshold, the target graphic element is displayed on the surface of the moving object in the real-time image;

[0079] If the overlapping area between the fixed object and the moving object does not reach a preset overlapping threshold, the target graphic element is displayed on the surface of the fixed object in the real-time image.

[0080] The fixed object includes the fixed parts in the medical radiation instrument, which can be the cover of the ionization chamber, and the moving object includes the patient to be examined. It is understandable that during the radiological examination, whether the patient's standing position is accurate has a great impact on the film output effect, so the patient needs to be guided to stand correctly before shooting.

[0081] Understandably, see Figure 4 , Figure 4 A relative position relationship diagram of the human body surface and the housing surface relative to the beam limiter or camera is provided in an embodiment of the present invention. Since the camera is on one side of the beam limiter, rather than in the center, this will result in that even if the patient stands in the center of the housing, the patient is not in the center of the housing from the camera's field of view. At this time, the image element will appear misplaced from the camera's field of view, and the image element will be displayed on the housing surface, which cannot provide effective guidance.

[0082] It should be noted that when the target primitive is displayed on the human body surface, it can accurately reflect the light field range, automatic exposure control ionization chamber and other positions. If the target primitive is displayed on the cover surface, the target primitive is blocked by the human body, and the complete target primitive cannot be observed in the visualization interface, and the patient cannot be guided to further stand, nor can more details of the part to be examined be obtained.

[0083] Therefore, it is necessary to identify the relative position of the cover and the patient. In an embodiment of the present invention, AI technology can be used to identify whether the human body reaches the front of the cover. Specifically, if the human body is in the center of the cover and the overlapping area with the cover reaches a preset overlapping threshold, for example, the overlapping area reaches eighty percent of the cover area, the target graphic element can be displayed on the human body surface in the real-time image; if the overlapping area between the human body and the cover does not reach the preset overlapping threshold, for example, the overlapping area does not reach eighty percent of the cover area, the target graphic element can be displayed on the cover surface in the real-time image to remind the doctor to guide the patient to stand accurately.

[0084] See also Figure 5 , Figure 5 This is a rendering of a target graphic element and a real-time image overlapped and displayed in a visualization interface provided in an embodiment of the present invention. It can be seen from the figure that the target graphic element is displayed on the surface of the human body, indicating that the patient has reached the front of the cover and the position is relatively accurate.

[0085] In addition, from Figure 5 The relative positions of the detector display area, light field range and ionization chamber can also be seen. Specifically, through the detector imaging range, doctors can observe the detector position blocked by the patient to prevent the patient's test part from being out of the detector's imaging range, resulting in unsatisfactory film output; through the automatic exposure control ionization chamber, doctors can directly select the measurement area of ​​the automatic exposure control ionization chamber on the real-time image to guide the patient's position. It should be noted that the automatic exposure control ionization chamber is a board in the physical coordinate system. When the emitted rays pass through the human body, they need to pass through this board. The purpose is to control the exposure time so that the rays reach different parts and different patients. The films have the same photosensitivity, making the film clearer. After the doctor selects the measurement area, he can guide the patient to place the part to be tested in the center of the measurement field to achieve the purpose of correct exposure.

[0086] In one embodiment of the present invention, the target image element includes a light field range; receiving an adjustment instruction for the target image element so that the adjustment execution device responds to the adjustment instruction and performs corresponding adjustment, comprising:

[0087] A drag adjustment instruction for the light field range is received, so that the adjustment execution device responds to the light field range adjustment instruction to adjust the size of the light field range.

[0088] The light field range is the area where the rays emitted by the beam limiter are radiated during radiological examination. In the embodiment of the present invention, the light field range is displayed in the single-frame exposure mode, and adjusting the light field range can be understood as adjusting the imaging area. In the traditional way, the imaging area is adjusted by operating the machine equipment in the exposure room, which is not only inconvenient, but also easy to produce waste films if the patient's position changes when the doctor leaves the exposure room.

[0089] In an embodiment of the present invention, when the target graphic element is displayed on the surface of the human body, that is, the light field range is displayed on the surface of the human body, the physician can drag and adjust the light field range in the visualization interface. Since the medical X-ray instrument is connected to the physician's operating terminal through communication, the medical X-ray instrument can automatically adjust the size of the beam limiter opening when receiving the adjustment instruction to achieve the adjustment of the light field range.

[0090] Furthermore, when the part to be inspected needs to be adjusted or a more accurate exposure image is needed for a part to be inspected, the region of interest can also be adjusted in real time by dragging the image element.

[0091] The camera can be used to assist doctors in adjusting the light field range and viewing the automatic exposure control measurement field on a visual interface, thereby determining whether the patient's position meets the shooting conditions and improving shooting efficiency.

[0092] In one embodiment of the present invention, the target graphic element further includes a splicing range and a movement range of the adjustment execution device; receiving an adjustment instruction for the target graphic element so that the adjustment execution device responds to the adjustment instruction and performs corresponding adjustment, including:

[0093] A drag adjustment instruction for the splicing range is received according to the movement range of the adjustment execution device, so that the adjustment execution device responds to the splicing range adjustment instruction to adjust the size of the splicing range.

[0094] Among them, the stitching range and the movement range of the adjustment execution device are not intuitively reflected in the actual system. In the embodiment of the present invention, the stitching range is displayed in the stitching exposure mode, and the stitching exposure mode is based on the single-frame exposure mode to move the adjustment execution device. It should be noted that the stitching exposure mode is generally used for patients who need to shoot a long part and cannot be fully displayed in one image. The stitching exposure mode will shoot a group of images, and then use the stitching algorithm to finally stitch this group of images into a visually smoother image.

[0095] The purpose of adjusting the motion range of the execution device is to prompt the doctor to adjust the extreme position of the motion of the execution device to prevent unreasonable setting of the stitching range. That is, before adjusting the stitching range, the corresponding number of stitching frames can be calculated in real time according to the motion range of the adjustment execution device, without the doctor manually setting the starting position and ending position of the adjustment execution device.

[0096] Specifically, when the doctor adjusts the stitching range in the visual interface of the operating terminal, that is, determines the imaging area, the image pixel coordinates in the imaging area can be converted into world coordinates; then during the exposure process, the beam limiter moves along a specific trajectory according to the stitching range set by the doctor after receiving the instruction, and the camera moves to the position of the first frame in sequence according to the world coordinates, takes an image, and then moves to the position of the second frame, takes the second image, and so on. Finally, the stitching algorithm is used to stitch this group of images into a visually smoother image.

[0097] The traditional workflow is optimized by making adjustments directly on the visual interface.

[0098] Based on the above interaction method, an embodiment of the present invention further provides an interaction device, and the interaction device corresponds one-to-one to the interaction method in the above embodiment.

[0099] See also Figure 6 , Figure 6 This is a schematic diagram of the structure of an interactive device provided by an embodiment of the present invention. The interactive device 600 includes an image acquisition module 601, a primitive drawing module 602, and a primitive adjustment module 603. The functional modules are described in detail as follows:

[0100] An image acquisition module 601 is used to acquire a real-time image including a target object;

[0101] The primitive drawing module 602 is used to draw the target primitive based on the target object, so that the target primitive and the real-time image are displayed in the visualization interface;

[0102] The primitive adjustment module 603 is used to receive an adjustment instruction for a target primitive, so that the adjustment execution device responds to the adjustment instruction and performs corresponding adjustment.

[0103] For the specific definition of the interactive device, please refer to the definition of the interactive method above, which will not be repeated here. Each module in the above interactive device can be implemented in whole or in part by software, hardware and a combination thereof. The above modules can be embedded in or independent of the processor in the electronic device in the form of hardware, or can be stored in the memory of the electronic device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0104] Based on the above interaction method, an embodiment of the present invention also provides an electronic device, including: a processor and a memory, and a computer program stored in the memory and executable on the processor; when the processor executes the computer program, the steps in the interaction method of the above embodiments are implemented.

[0105] Figure 77 is a schematic diagram of the structure of an electronic device 700 suitable for implementing the embodiment of the present invention. The electronic device in the embodiment of the present invention may include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle terminals (such as vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 7 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0106] The electronic device includes: a memory and a processor, wherein the processor here may be referred to as the processing device 701 described below, and the memory may include at least one of the read-only memory (ROM) 702, the random access memory (RAM) 703, and the storage device 708 described below, as shown below:

[0107] like Figure 7 As shown, the electronic device 700 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the electronic device 700 are also stored. The processing device 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0108] Typically, the following devices may be connected to the I / O interface 705: an input device 706 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 707 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 708 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 709. The communication device 709 may allow the electronic device 700 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 7 The electronic device 700 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0109] In particular, according to an embodiment of the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 709, or installed from the storage device 708, or installed from the ROM 702. When the computer program is executed by the processing device 701, the above-mentioned functions defined in the method of the embodiment of the present invention are executed.

[0110] Based on the above-mentioned interaction method, an embodiment of the present invention also provides a computer-readable storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the steps in the interaction method of the above-mentioned embodiments.

[0111] Those skilled in the art will appreciate that all or part of the processes of the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, wherein the computer-readable storage medium is a disk, an optical disk, a read-only storage memory, or a random access memory, etc.

[0112] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.

Claims

1. An interactive method, characterized in that: include: Acquire a real-time image including a target object; Acquire depth information of the real-time image, and construct a three-dimensional space coordinate system based on the depth information, acquire first coordinate information of the target object in the three-dimensional space coordinate system, convert the first coordinate information in the three-dimensional space coordinate system into second coordinate information in a two-dimensional space coordinate system based on a preset dimension conversion relationship, and draw the target primitive based on the second coordinate information so that the target primitive and the real-time image are displayed in a visualization interface, wherein the target primitive is a semi-transparent basic graphic; An adjustment instruction for the target graphic element is received, so that the adjustment execution device responds to the adjustment instruction and performs corresponding adjustment, wherein the adjustment instruction comes from the physician operation terminal.

2. The method according to claim 1, characterized in that Determining the preset dimension conversion relationship includes: Establishing an association relationship between the three-dimensional space coordinate system and the two-dimensional space coordinate system by using a preset calibration algorithm; The preset dimension conversion relationship is determined according to the association relationship.

3. The method according to claim 1, characterized in that The step of displaying the target graphic element and the real-time image in a visualization interface includes: The target graphic element is overlapped with the real-time image and displayed in a visualization interface, and the target graphic element is displayed on the real-time image.

4. The method according to claim 3, characterized in that The real-time image also includes a fixed object and a moving object; and displaying the target image element on the real-time image includes: identifying a relative position of the fixed object and the moving object; If the overlapping area between the fixed object and the moving object reaches a preset overlapping threshold, displaying the target graphic element on the surface of the moving object in the real-time image; If the overlapping area between the fixed object and the moving object does not reach a preset overlapping threshold, the target graphic element is displayed on the surface of the fixed object in the real-time image.

5. The method according to claim 4, characterized in that The target image element includes a light field range; the receiving of an adjustment instruction for the target image element so that the adjustment execution device responds to the adjustment instruction and performs corresponding adjustment, comprises: A drag adjustment instruction for the light field range is received, so that the adjustment execution device adjusts the size of the light field range in response to the light field range adjustment instruction.

6. The method according to claim 4, characterized in that The target graphic element also includes a splicing range and a movement range of the adjustment execution device; the receiving of the adjustment instruction for the target graphic element so that the adjustment execution device responds to the adjustment instruction and performs corresponding adjustment, includes: A drag adjustment instruction for the splicing range is received according to the movement range of the adjustment execution device, so that the adjustment execution device adjusts the size of the splicing range in response to the splicing range adjustment instruction.

7. An interactive device, characterized in that: include: An image acquisition module, used to acquire a real-time image including a target object; A primitive drawing module, used to obtain depth information of the real-time image, and construct a three-dimensional space coordinate system based on the depth information, obtain first coordinate information of the target object in the three-dimensional space coordinate system, convert the first coordinate information in the three-dimensional space coordinate system into second coordinate information in a two-dimensional space coordinate system based on a preset dimension conversion relationship, and draw the target primitive based on the second coordinate information, so that the target primitive and the real-time image are displayed in a visualization interface, and the target primitive is a semi-transparent basic graphic; The graphic element adjustment module is used to receive an adjustment instruction for the target graphic element so that the adjustment execution device responds to the adjustment instruction and performs corresponding adjustment, and the adjustment instruction comes from the physician operation terminal.

8. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the interaction method according to any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the interaction method according to any one of claims 1 to 6 are implemented.

Images