Interaction system and method thereof
By designing an interactive system including carrier module, click module, tracking module, conversion module and judgment module, the problem of reduced sensitivity and poor operation fluency of medical devices in the hospital operating room is solved, and the functional improvement of remote interaction functions and operations is achieved.
Patent Information
- Application Number
- CN202510128001.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-02-05
- Publication Date
- 2025-05-13
AI Technical Summary
When the medical devices in the hospital operating room are operated in a sterile environment, the touch screen sensitivity of the display screen is reduced, the operation fluency becomes worse, and the doctor's surgical position is far away from the operating screen, so the software functions cannot be operated at any time.
Design an interactive system, including carrier module, click module, tracking module, conversion module and judgment module, realize remote interaction function through the positional relationship between the click module and the carrier module, and add sliding operations and enable operations on the basis of ordinary key operations.
It realizes interactive functions without directly operating the screen, improves the functionality and smoothness of the operation, avoids misoperation, and is suitable for medical device operations in sterile environments.
Smart Images
Figure CN119987634A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of medical devices, and in particular to an interactive system and method thereof. Background Art
[0002] At present, medical equipment in hospital operating rooms needs to be operated in a sterile environment, so medical staff need to bag the display screen. The sterile bag will reduce the touch screen sensitivity of the operation screen, the operation fluency will be worse, and the doctor's operating position is often far away from the operation screen, and the software functions cannot be operated at any time.
[0003] Therefore, there is an urgent need for a solution that enables interaction without operating on the operation screen, so that doctors can implement corresponding interactive functions at any time. Summary of the invention
[0004] Purpose of the invention: In view of the above-mentioned deficiencies, the present invention proposes an interactive system and method thereof, which can realize remote interactive functions. In addition, the interaction of the present invention adds sliding operations on the basis of ordinary key operations, thereby enriching the functionality of the operations.
[0005] Technical solution: The present invention provides an interactive system, comprising: A carrier module, on which at least one interactive area is provided, and each interactive area is provided with a corresponding interactive function; A click module, on which an interaction end is provided, for making contact with any interaction area on the carrier module; The tracking module comprises a first tracing component and a second tracing component and a tracking component respectively arranged on the carrier module and the click module; the tracking component is used to obtain the real-time position and posture of the carrier module and the click module according to the first tracing component and the second tracing component respectively; A conversion module, used to convert the real-time local range of each interactive area according to the real-time posture of the carrier module and the positional relationship of the interactive area relative to the carrier module, and to convert the real-time position of the interactive terminal according to the real-time posture of the click module and the positional relationship of the interactive terminal relative to the click module; The judgment module is used to judge whether the real-time position of the interaction terminal is within the real-time local range of a certain interaction area; if so, the interaction function corresponding to the interaction area is triggered; if not, the interaction function corresponding to the interaction area is not triggered.
[0006] Specifically, the carrier module is configured as an interactive panel, and a plurality of interactive areas are arranged on an operating surface of the interactive panel.
[0007] More specifically, each of the interaction areas is independent and non-continuous on the interaction panel.
[0008] More specifically, the outline shape of the interaction area on the interaction panel is set to a regular geometric shape.
[0009] More specifically, the interactive panel is arranged on an array plane where an optical tracing array formed by a patient tracer is located. At this time, the first tracing assembly and the patient tracer share an optical tracing element.
[0010] More specifically, when the tracking component identifies the first tracer component, a carrier coordinate system is established to obtain the position and posture of the carrier module. When the interaction area is arranged in an array, the two coordinate axes in the carrier coordinate system are respectively set to be parallel to the rectangular boundaries of the interaction area.
[0011] Specifically, the click module is configured as an interactive probe having a first end that can contact the interactive area, ie, an interactive end.
[0012] More specifically, when the tracking component identifies the second tracer component, a probe coordinate system is established to obtain the position of the interactive probe. When the axis of the interactive probe passes through the plane where the second tracer component is located, a coordinate axis of the probe coordinate system is set to be parallel to the probe axis.
[0013] Specifically, when the click module contacts the carrier module, a determination rule for the positional relationship between the interaction terminal of the click module and the interaction area is set, thereby determining that the real-time position of the interaction terminal is within the real-time local range of a certain interaction area, thereby completing the triggering action.
[0014] More specifically, the judgment module, in addition to identifying whether the real-time position of the interaction terminal is within the real-time local range of a certain interaction area, also includes determining the interaction area closest to the interaction terminal, specifically by calculating the distance between the interaction terminal and the center points of all interaction areas and screening out the interaction area corresponding to the smallest distance.
[0015] Specifically, the contact between the click module and the carrier module includes instantaneous / continuous static point contact, or continuous dynamic point contact to form a motion track on the carrier module.
[0016] More specifically, the interactive area on the carrier module includes an enabling area, a sliding operation area and a plurality of button areas; The enabling area is used by the determination module to determine whether to enable the interactive operation; The sliding operation area is used for the interactive end to generate continuous dynamic point contact thereon to trigger the corresponding function; The key area is used for the interactive module to generate instantaneous / continuous static point contact thereon to trigger a corresponding function.
[0017] Furthermore, the button area is set as a rectangular area, and the judgment module judges the positional relationship between the interaction terminal and the rectangular area, including that the interaction terminal is located within the rectangular area, the interaction terminal is located on the edge of the rectangular area, or the interaction terminal is located outside the rectangular area.
[0018] Furthermore, the connection lines between the interactive terminal and the four corner points of the rectangular area are obtained, and the angles between each connection line and each side of the rectangular area are calculated; If they are all acute angles, the interaction end is located within the rectangular area; If there is an obtuse angle, the end of the interaction is outside the rectangular area; If there is a 0° angle, the interaction end is located at the edge of the rectangular area.
[0019] Furthermore, after obtaining the set time that the interaction terminal stays in the sliding operation area, the judgment module records the current position of the interaction terminal as the initial position. When the position of the interaction terminal changes and the range of the change exceeds the set threshold, the judgment module compares the real-time position of the interaction terminal with its initial position to obtain its moving direction, thereby triggering the function corresponding to the moving direction.
[0020] Furthermore, the two short boundaries of the sliding operation area form upper and lower boundaries on the carrier module, and the direction vectors of the center point of the sliding operation area pointing to the upper and lower boundaries along the length direction of the sliding operation area are respectively obtained. Then, by judging whether the vector between the real-time position of the interaction terminal and its initial position is in the same direction as the aforementioned direction vector, it is determined whether the interaction terminal is moving toward the upper limit or lower limit of the sliding operation area.
[0021] Furthermore, by judging whether the angle between the vector between the real-time position of the interactive terminal and its initial position and the direction vector is less than the set angle, if so, the two are considered to be in the same direction, otherwise, the two are in different directions.
[0022] Furthermore, the enabling area is set as a point on the carrier module, and the judgment module judges whether the interactive terminal is located within a range where the distance from the point corresponding to the enabling area is less than a set threshold. If so, it is considered that the interactive terminal stays in the enabling area.
[0023] Furthermore, when the interactive terminal is placed in other areas, the judgment module judges to open the operation of the corresponding area and close the current operation.
[0024] Specifically, it also includes a display module, which is connected to the judgment module and displays the interactive operation triggered by the judgment module in real time for observation by operators.
[0025] Specifically, the tracking module adopts the optical tracking principle to obtain the real-time position and posture of the carrier module and the click module.
[0026] The present invention also provides an interactive method using the aforementioned interactive system, comprising: S1, the tracking component obtains the real-time position and posture of the carrier module and the click module according to the first tracing component and the second tracing component respectively; S2, the conversion module converts the real-time local range of each interactive area according to the real-time posture of the carrier module and the positional relationship of the interactive area relative to the carrier module, and converts the real-time position of the interactive terminal according to the real-time posture of the click module and the positional relationship of the interactive terminal relative to the click module; S3. The judgment module judges whether the real-time position of the interaction terminal is within the real-time local range of each interaction area; if so, the interaction function corresponding to the interaction area is triggered; if not, the interaction function corresponding to the interaction area is not triggered.
[0027] Beneficial effects: The present invention uses the positional relationship between the click module and the carrier module as the interactive interface, obtains the real-time local range of each interactive area and the real-time position of the interactive terminal through the tracking module, and then determines whether the real-time position of the interactive terminal is within the real-time local range of each interactive area through the judgment module, and triggers the interactive function corresponding to the interactive area when it is within the real-time local range, so as to realize the remote interactive function. In addition, the interaction of the present invention adds sliding operation on the basis of ordinary key operation, enriches the functionality of the operation, and adds enabling operation, which can effectively avoid misoperation. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] In order to more clearly illustrate the technical solutions in the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are merely embodiments of the present invention, and ordinary technicians in this field can obtain other drawings based on these drawings without creative work.
[0029] Figure 1 is an architecture diagram of the interactive system of the present invention; Figure 2 is a structural example diagram of a carrier module of the present invention; Figure 3 An example diagram for determining the positional relationship between the probe tip and the rectangular area; Figure 4 A flowchart for performing a sliding operation; Figure 5 To obtain an example diagram of the transformation relationship between the probe tip and the interaction area on the carrier module; Figure 6 It is a flow chart of the interactive method of the present invention.
[0030] Among them, 10 is an interactive panel, 11 is an optical tracking element, 12 is an interactive area, 121 is a button area, 122 is a sliding operation area, 123 is an enabling area, and 13 is a fixing part. DETAILED DESCRIPTION
[0031] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present application is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.
[0032] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present invention should have the usual meanings understood by people with ordinary skills in the field to which the present invention belongs. "Include" or "comprising" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect.
[0033] The architecture of the interactive system of the present invention is as follows Figure 1 As shown, including: A carrier module, on which at least one interactive area is provided, and each interactive area is provided with a corresponding interactive function; A click module, on which an interaction end is provided, for making contact with any interaction area on the carrier module; The tracking module comprises a first tracking component, a second tracking component and a tracking component; the first tracking component is arranged on the carrier module, and the second tracking component is arranged on the click module; the tracking component is used to obtain the real-time posture of the carrier module and the click module according to the first tracking component and the second tracking component respectively; A conversion module, used to convert the real-time local range of each interactive area according to the real-time posture of the carrier module and the positional relationship of the interactive area relative to the carrier module, and to convert the real-time position of the interactive terminal according to the real-time posture of the click module and the positional relationship of the interactive terminal relative to the click module; The judgment module is used to judge whether the real-time position of the interaction terminal is within the real-time local range of a certain interaction area; if so, the interaction function corresponding to the interaction area is triggered; if not, the interaction function corresponding to the interaction area is not triggered.
[0034] In the present invention, the tracking module can adopt the optical tracking principle to obtain the real-time posture of the carrier module and the click module. In this implementation mode, the first tracking component and the second tracking component can be set as an optical tracking array, and the optical tracking array includes a plurality of optical tracking elements 11. The tracking module can be set as an optical tracking system.
[0035] Specifically, the first tracking component includes at least three non-collinear optical tracking elements 11, and the optical tracking elements 11 are arranged on any plane of the carrier module. The optical tracking system can obtain the optical coordinates of all optical tracking elements 11 in real time, and obtain the real-time posture of the carrier module according to the optical coordinates of all optical tracking elements 11.
[0036] Exemplarily, the carrier module can be constructed as an interactive panel 10, and a plurality of interactive areas 12 are provided on the operating surface of the interactive panel 10. In order to simplify the structural design, the interactive panel 10 can be provided on the array plane where the optical tracer array on the patient is located. At this time, the optical tracer array formed by the first tracer component and the patient tracer shares the optical tracer element. It should be noted that the patient tracer is generally fixed to the patient's anatomical structure, and the optical tracking of the patient tracer by the optical tracking system is used to align the anatomical structure in the medical image with the actual anatomical structure of the patient, thereby helping the doctor to perform image-guided surgery. Therefore, in this embodiment, the patient tracer is the first tracer component, and by tracking the patient tracer by the optical tracking system, the real-time position and posture of the anatomical structure under the optical tracking system and the real-time position and posture of the interactive panel under the optical tracking system can be obtained at one time.
[0037] In addition, the outline shape of the interactive area on the interactive panel 10 can be set to a regular geometric shape, including a rectangle, a circle, an ellipse, etc., without limitation. This embodiment is described using a rectangle as an example.
[0038] In order to facilitate the identification of whether the real-time position of the interactive terminal is within the real-time local range of each interactive area 12 and trigger the corresponding function of the interactive area 12, the interactive area 12 is arranged on the interactive panel 10 in an array. Specifically, when the optical tracking system identifies the optical tracking element 11 of the first tracking component, the carrier coordinate system can be established to obtain the position and posture of the carrier module. When the interactive area 12 is arranged in an array, the two coordinate axes in the carrier coordinate system can be set to be parallel to the boundaries of the rectangle corresponding to the interactive area 12. In this case, the position relationship of the interactive area 12 relative to the carrier module can be quickly calculated, and the real-time local range of each interactive area 12 can be converted to improve the conversion efficiency.
[0039] Furthermore, each interaction area 12 is independent and non-continuous on the interaction panel 10 .
[0040] In the present invention, the judgment module not only identifies whether the real-time position of the interaction terminal is within the real-time local range of a certain interaction area 12, but also includes determining the interaction area 12 closest to the interaction terminal. The specific determination method is as follows: the distance between the interaction terminal and the center point of all interaction areas 12 can be calculated and the interaction area 12 corresponding to the smallest distance can be obtained by screening.
[0041] Exemplarily, the click module can be constructed as an interactive probe, and the interactive probe has a first end that can contact the interactive area, that is, the interactive end, which is generally a probe tip. The second tracking component is set at any position of the interactive probe, for example, an optical tracking array with optical tracking elements is constructed at the second end of the interactive probe, and the axis of the interactive probe passes through the plane of the optical tracking array. Specifically, the second tracking component includes at least three non-collinear optical tracking elements, and the optical tracking system can obtain the optical coordinates of all optical tracking elements in real time and obtain the real-time position and posture of the interactive probe. Specifically, after the optical tracking system identifies the second tracking element, it can establish a probe coordinate system to obtain the position and posture of the interactive probe. When the axis of the interactive probe passes through the plane of the optical tracking array, that is, the interactive probe and the optical tracking array are coplanar, a certain coordinate axis of the established carrier coordinate system can be set to be parallel to the probe axis. In this case, the position of the first end relative to the interactive probe can be quickly calculated to determine the real-time position of the first end of the interactive probe, so as to quickly determine whether the contact position of the first end of the interactive probe and the interactive panel is located within the real-time local range of a certain interactive area 12.
[0042] In the present invention, the contact between the click module and the carrier module includes point contact, and it is only necessary to determine whether the contact point is within the real-time local range of the interactive area to form an instruction on whether the function is triggered. It can be imagined that the point contact mentioned here is not limited to point contact in the literal sense, and can be line contact or surface contact. It is only necessary to set the judgment rule of the positional relationship between the interactive end of the click module and the interactive area when contacting, so as to determine that the real-time position of the interactive end is within the real-time local range of each interactive area, and then complete the triggering action.
[0043] Exemplarily, the point contact between the click module and the carrier module can be instantaneous / continuous contact, such as instantaneous / continuous static point contact, or can be continuous dynamic point contact to form a motion track on the carrier module, corresponding to triggering different functions.
[0044] Furthermore, if the contact between the click module and the carrier module adopts continuous dynamic point contact, it is also necessary to determine whether the motion trajectory formed by the interactive end of the click module in the interactive area corresponding to the carrier module is all located in the interactive area. The specific determination rules are such as the determination rules for the positional relationship between the interactive end of the click module and the interactive area during the contact set as mentioned above.
[0045] In the present invention, reference is made to Figure 2, the interactive area 12 on the carrier module may include several button areas 121, each of which will have a corresponding text description and a separate boundary line, which are non-continuous with each other and correspond to different interactive operations. The specific interactive functions implemented by each button area 121 are determined according to actual needs. Further, after determining that the real-time position of the interactive terminal is within the real-time local range of a certain interactive area, the judgment module can determine whether the interactive terminal has made an instantaneous or continuous static contact with the corresponding button area 121 according to the time the interactive terminal stays in the corresponding button area 121. Specifically, the judgment is made according to the stay time of the interactive terminal, which can be set to: the stay time of instantaneous static contact is set to 1-2 seconds, and the stay time of continuous static contact is set to more than 2 seconds.
[0046] In this embodiment, the interactive functions achieved by instantaneous static contact can be set according to actual needs, such as switching view size, fine-tuning, view switching, switching channels or pages, etc. The interactive functions achieved by continuous static contact can be set according to actual needs, such as resetting the current channel or robotic arm execution, etc.
[0047] In this embodiment, refer to Figure 2 The button area 121 is set as a rectangular area, and several button areas 121 can be evenly distributed on the interactive panel 10. The judgment module judges the positional relationship between the interactive terminal and the rectangular area generally including that the interactive terminal is located in the rectangular area, the interactive terminal is located on the edge of the rectangular area, or the interactive terminal is located outside the rectangular area.
[0048] In this embodiment, the determination module determines the positional relationship between the interactive terminal and the rectangular area as follows: Figure 3 As shown, the details are as follows: Get the connecting lines between the interactive terminal P and the four corner points P1, P2, P3, and P4 of the rectangular area, and calculate the angles between each connecting line and each side of the rectangular area; if they are all acute angles, the interactive terminal is located in the rectangular area; if there is an obtuse angle, the interactive terminal is located outside the rectangular area; if there is a 0° angle, the interactive terminal is located on the edge of the rectangular area.
[0049] In the present invention, continue to refer to Figure 2, the interactive area 12 on the carrier module may include a sliding operation area 122, which is specifically a long strip groove, which is also equipped with text and sliding direction instructions. The length, width and depth of the groove are all larger than the size of the interactive terminal, and can completely cover the interactive terminal. The interactive terminal can slide smoothly and unimpeded in the groove to generate continuous dynamic contact for realizing the corresponding sliding operation. In this embodiment, the interactive functions realized by the corresponding sliding operation include window position and width adjustment, zoom adjustment or layer adjustment. In this embodiment, after obtaining the set time (such as 1-2s) that the interactive terminal stays in the sliding operation area 122, the judgment module determines that the real-time position of the interactive terminal is in the sliding operation area 122, thereby judging that a sliding operation is to be performed. At this time, the judgment module records the current position of the interactive terminal as the initial position, which corresponds to the initial position of sliding in the sliding operation area 122 after the current function is turned on. When the position of the interactive terminal changes, and the range of the change exceeds a set threshold (such as 1-2 mm), the judgment module compares the real-time position of the interactive terminal with its initial position to obtain its moving direction, that is, whether it moves to the upper limit of the sliding operation area 122 or to the lower limit of the sliding operation area 122, thereby performing a corresponding upward or downward sliding operation, such as Figure 4 When the judgment module determines that the interactive terminal leaves the sliding operation area 122, it is determined that the sliding operation is finished; if the judgment module determines that the interactive terminal enters the sliding operation area 122 again, the above judgment is repeated.
[0050] Specifically, the determination module determines whether the real-time position of the interaction terminal moves toward the upper limit of the sliding operation area 122 or toward the lower limit of the sliding operation area 122 compared with its initial position, as follows: The two short boundaries of the sliding operation area 122 form upper and lower boundaries on the carrier module. The direction vectors of the center point of the sliding operation area 122 pointing to the upper and lower boundaries along the length direction of the sliding operation area 122 are respectively obtained. It can be determined whether the vector between the real-time position of the interaction terminal and its initial position is in the same direction as the aforementioned direction vector, thereby determining whether the interaction terminal is moving toward the upper limit or lower limit of the sliding operation area 122.
[0051] More specifically, it can be determined by judging whether the angle between the vector between the real-time position of the interaction terminal and its initial position and the aforementioned direction vector is less than a set angle, such as 2°. If so, it is considered that the vector between the real-time position of the interaction terminal and its initial position is in the same direction as the aforementioned direction vector; otherwise, they are not in the same direction.
[0052] Furthermore, the coordinates of the center point of the sliding operation area 122 can be measured by three-coordinate measurement, with the upper or lower boundary of the sliding operation area 122 as the x-axis, the long boundary of the sliding operation area 122 as the y-axis, and the plane perpendicular to the sliding operation area 122 as the z-axis, to establish a corresponding coordinate system. Thus, the coordinates of the corner points of the sliding operation area 122 can be calculated based on the design parameters of the sliding operation area 122, and the direction vector of the center point of the sliding operation area 122 pointing to the upper and lower boundaries along the length direction of the sliding operation area 122 can be calculated.
[0053] In the present invention, continue to refer to Figure 2 , the interactive area 12 on the carrier module may also include an enabling area 123, which is also equipped with clearly visible boundary lines and text descriptions. The enabling area 123 is used for the judgment module to judge whether to turn on the interactive operation. Specifically, the judgment module can determine whether to turn on the interactive operation based on the interactive terminal staying in the enabling area 123 for a set time, that is, judging that the real-time position of the interactive terminal is in the enabling area 123. At this time, the judgment module will subsequently judge whether the real-time position of the interactive terminal is within the real-time local range of each interactive area; similarly, when the interactive terminal stays in the enabling area 123 for a set time again, it is determined to turn off the interactive operation. Through the setting of the enabling area 123, the function confusion caused by the click module being accidentally touched can be effectively prevented, and the calculation amount of the judgment module can also be reduced. In this embodiment, the set time is set to 1-2 seconds.
[0054] In the present invention, the enabling area 123 may be a circular area disposed on the carrier module.
[0055] In this embodiment, the enabling area 123 may be a point set on the interactive panel 10, and the judgment module determines whether the interactive terminal stays in the enabling area 123. Specifically, the judgment module determines whether the interactive terminal is located within a range where the distance from the point corresponding to the enabling area 123 is less than a set threshold. If so, it is considered that the interactive terminal stays in the enabling area 123.
[0056] In the present invention, when the judgment module judges that the interactive terminal is placed in other areas, the judgment module judges to open the interactive operation of the corresponding area and close the current interactive operation.
[0057] Specifically, in this embodiment, a fixing member 13 may be fixed at the bottom of the interactive panel 10 to fix and install the interactive panel 10 .
[0058] In the present invention, the conversion module obtains the relative position relationship between the interactive terminal and the second tracer component according to the measurement, calculates the transformation relationship between the interactive terminal and the second tracer component, and then obtains the position relationship between the interactive terminal and the click module according to the installation parameters of the second tracer component on the click module; Specifically, the relative position relationship between the interactive terminal and the second tracking component can be obtained through three-coordinate measurement. According to the measured relative position relationship of the optical tracking elements in the second tracking component, a click module coordinate system can be established, thereby calculating the coordinates of the interactive terminal in the click module coordinate system, thereby generating a Rom file and storing it in the corresponding json file.
[0059] The conversion module obtains the relative position relationship between the first tracer component on the carrier module and the center point position of each interactive area 12 by measurement, calculates the transformation relationship between the first tracer component and the interactive area 12, and then obtains the position relationship of the interactive area 12 relative to the carrier module according to the installation parameters of the first tracer component on the carrier module; Specifically, the relative position relationship between the first tracking component on the carrier module and the center point position of each interaction area 12 can be obtained through three-coordinate measurement. According to the measured relative position relationship of the optical tracking element 11 in the first tracking component, a carrier coordinate system is established, thereby calculating the coordinates of the center point of each interaction area 12 in the carrier coordinate system, thereby generating a Rom file and storing it in the corresponding json file.
[0060] In the present invention, the conversion module can intermittently send information acquisition requests to the tracking module in a polling manner (e.g., using 33 frames per second), and the tracking module obtains the real-time position and posture of the first tracer component and the second tracer component according to the request, and stores it in the data storage module. The first tracer component and the second tracer component both have unique identifiers, and the conversion module reads the real-time position and posture of the relevant tracer component stored in the data storage module, and then can be combined with the position relationship of the interactive area relative to the carrier module to convert the real-time local range of each interactive area, and can also be combined with the position relationship of the interactive terminal relative to the click module to convert the real-time position of the interactive terminal.
[0061] Specifically, the conversion module reads the real-time position and posture of the relevant tracer component stored in the data storage module, and can obtain the real-time position transformation relationship T of the first tracer component relative to the tracking component in real time through the identification index. p1 The real-time position transformation relationship T between the second tracer component and the tracking component p2 , so that the transformation relationship T of the first tracer component relative to the second tracer component can be obtained p3 =(T p2 ) -1 T p1 Then, the conversion module combines the position relationship T1 of the interaction area relative to the carrier module and the position relationship T2 of the interaction terminal relative to the click module to calculate the transformation relationship T3=T1T between the interaction terminal and the interaction area on the carrier module. p3(T2) -1 ,like Figure 5 shown.
[0062] The present invention also includes a display module, which is connected to the judgment module and displays the interactive operation triggered by the judgment module in real time for observation by operators.
[0063] The present invention also provides an interactive method based on the aforementioned interactive system, such as Figure 6 As shown, the steps include: S1, the tracking module is connected, and the tracking component obtains the real-time position and posture of the carrier module and the click module according to the first tracking component and the second tracking component respectively; S2, the conversion module converts the real-time local range of each interactive area according to the real-time posture of the carrier module and the positional relationship of the interactive area relative to the carrier module, and converts the real-time position of the interactive terminal according to the real-time posture of the click module and the positional relationship of the interactive terminal relative to the click module; S3. The judgment module judges whether the real-time position of the interaction terminal is within the real-time local range of a certain interaction area; if so, the interaction function corresponding to the interaction area is triggered; if not, the interaction function corresponding to the interaction area is not triggered.
[0064] The present invention further includes S4, where the display module displays the interactive operation triggered by the judgment module in real time.
[0065] The present invention uses the positional relationship between the click module and the carrier module as the interactive interface, obtains the real-time local range of each interactive area and the real-time position of the interactive terminal through the tracking module, and then determines whether the real-time position of the interactive terminal is within the real-time local range of each interactive area through the judgment module, and triggers the interactive function corresponding to the interactive area when it is within the range, so as to realize the remote interactive function. In addition, the interaction of the present invention adds sliding operation on the basis of ordinary key operation, enriches the functionality of the operation, and adds enabling operation, which can effectively avoid misoperation.
[0066] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present invention (including the claims) is limited to these examples. Under the concept of the present invention, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present invention as described above, which are not provided in detail for the sake of simplicity.
[0067] The embodiments of the present invention are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present invention should be included in the protection scope of the present invention.
Claims
1. An interactive system, characterized in that include: A carrier module, on which at least one interactive area is provided, and each interactive area is provided with a corresponding interactive function; A click module, on which an interaction end is provided, for making contact with any interaction area on the carrier module; The tracking module comprises a first tracing component and a second tracing component and a tracking component respectively arranged on the carrier module and the click module; the tracking component is used to obtain the real-time position and posture of the carrier module and the click module according to the first tracing component and the second tracing component respectively; A conversion module, used to convert the real-time local range of each interactive area according to the real-time posture of the carrier module and the positional relationship of the interactive area relative to the carrier module, and to convert the real-time position of the interactive terminal according to the real-time posture of the click module and the positional relationship of the interactive terminal relative to the click module; The judgment module is used to judge whether the real-time position of the interaction terminal is within the real-time local range of a certain interaction area; if so, the interaction function corresponding to the interaction area is triggered; if not, the interaction function corresponding to the interaction area is not triggered.
2. The interactive system according to claim 1, characterized in that: The carrier module is configured as an interactive panel, and a plurality of interactive areas are arranged on an operating surface of the interactive panel.
3. The interactive system according to claim 2, characterized in that: Each of the interaction areas is independent and non-continuous on the interaction panel.
4. The interactive system according to claim 2, characterized in that: The outline shape of the interaction area on the interaction panel is set to a regular geometric shape.
5. The interactive system according to claim 2, characterized in that: The interactive panel is arranged on an array plane where an optical tracing array formed by a patient tracer is located. At this time, the first tracing component and the patient tracer share an optical tracing element.
6. The interactive system according to claim 2, characterized in that: When the tracking component identifies the first tracer component, a carrier coordinate system is established to obtain the position and posture of the carrier module. When the interaction area is arranged in an array, the two coordinate axes in the carrier coordinate system are set to be parallel to the rectangular boundaries of the interaction area.
7. The interactive system according to claim 1, characterized in that: The click module is configured as an interactive probe having a first end, ie, an interactive end, which can contact the interactive area.
8. The interactive system according to claim 7, characterized in that: When the tracking component identifies the second tracer component, a probe coordinate system is established to obtain the position and posture of the interactive probe. When the axis of the interactive probe passes through the plane where the second tracer component is located, a coordinate axis of the probe coordinate system is set to be parallel to the probe axis.
9. The interactive system according to claim 1, characterized in that: When the click module contacts the carrier module, a determination rule for the positional relationship between the interaction terminal of the click module and the interaction area is set, thereby determining that the real-time position of the interaction terminal is within the real-time local range of a certain interaction area, thereby completing the triggering action.
10. The interactive system according to claim 9, characterized in that: The judgment module, in addition to identifying whether the real-time position of the interaction terminal is within the real-time local range of a certain interaction area, also includes determining the interaction area closest to the interaction terminal, specifically by calculating the distance between the interaction terminal and the center points of all interaction areas and screening out the interaction area corresponding to the smallest distance.
11. The interactive system according to claim 1, characterized in that: The contact between the click module and the carrier module includes instantaneous / continuous static point contact, or continuous dynamic point contact to form a motion track on the carrier module.
12. The interactive system according to claim 11, characterized in that: The interactive area on the carrier module includes an enabling area, a sliding operation area and a plurality of button areas; The enabling area is used by the determination module to determine whether to enable the interactive operation; The sliding operation area is used for the interactive end to generate continuous dynamic point contact thereon to trigger the corresponding function; The key area is used for the interactive module to generate instantaneous / continuous static point contact thereon to trigger a corresponding function.
13. The interactive system according to claim 12, characterized in that: The button area is set as a rectangular area, and the judgment module judges that the positional relationship between the interaction terminal and the rectangular area includes that the interaction terminal is located in the rectangular area, the interaction terminal is located on the edge of the rectangular area, or the interaction terminal is located outside the rectangular area.
14. The interactive system according to claim 13, characterized in that: Obtaining the connecting lines between the interactive terminal and the four corner points of the rectangular area, and calculating the angles between each connecting line and each side of the rectangular area; If they are all acute angles, the interaction end is located within the rectangular area; If there is an obtuse angle, the end of the interaction is outside the rectangular area; If there is a 0° angle, the interaction end is located at the edge of the rectangular area.
15. The interactive system according to claim 12, characterized in that: After obtaining the set time that the interaction terminal stays in the sliding operation area, the judgment module records the current position of the interaction terminal as the initial position. When the position of the interaction terminal changes and the range of the change exceeds the set threshold, the judgment module compares the real-time position of the interaction terminal with its initial position to obtain its moving direction, thereby triggering the function corresponding to the moving direction.
16. The interactive system according to claim 15, characterized in that: The two short boundaries of the sliding operation area form upper and lower boundaries on the carrier module, and the direction vectors of the center point of the sliding operation area pointing to the upper and lower boundaries along the length direction of the sliding operation area are respectively obtained. By judging whether the vector between the real-time position of the interaction terminal and its initial position is in the same direction as the aforementioned direction vector, it is determined whether the interaction terminal is moving toward the upper limit or lower limit of the sliding operation area.
17. The interactive system according to claim 16, characterized in that: By judging whether the angle between the vector between the real-time position of the interactive terminal and its initial position and the direction vector is less than the set angle, if so, it is considered that the two are in the same direction, otherwise, the two are in different directions.
18. The interactive system according to claim 12, characterized in that: The enabling area is set as a point on the carrier module, and the judgment module judges whether the interactive terminal is located within a range where the distance from the point corresponding to the enabling area is less than a set threshold. If so, it is considered that the interactive terminal stays in the enabling area.
19. The interactive system according to claim 12, characterized in that: When the interactive terminal is placed in other areas, the judgment module judges to open the operation of the corresponding area and close the current operation.
20. The interactive system according to claim 1, characterized in that: It also includes a display module, which is connected to the judgment module and displays the interactive operation triggered by the judgment module in real time for observation by operators.
21. The interactive system according to claim 1, characterized in that: The tracking module uses the optical tracking principle to obtain the real-time positions and postures of the carrier module and the click module.
22. An interactive method using the interactive system according to any one of claims 1 to 21, characterized in that: include: S1, the tracking component obtains the real-time position and posture of the carrier module and the click module according to the first tracing component and the second tracing component respectively; S2, the conversion module converts the real-time local range of each interactive area according to the real-time posture of the carrier module and the positional relationship of the interactive area relative to the carrier module, and converts the real-time position of the interactive terminal according to the real-time posture of the click module and the positional relationship of the interactive terminal relative to the click module; S3. The judgment module judges whether the real-time position of the interaction terminal is within the real-time local range of each interaction area; if so, the interaction function corresponding to the interaction area is triggered; if not, the interaction function corresponding to the interaction area is not triggered.