Self-adaptive guiding system and method based on patient posture positioning

Abstract

The invention relates to the technical field of medical auxiliary equipment, and discloses a self-adaptive guiding system and method based on patient posture positioning, and the system comprises a multi-modal sensing module which obtains a skeleton key point sequence, body pressure distribution data and human eye vision parameters of a patient; the intelligent processing module is used for calculating the real-time posture of the patient and matching the real-time posture with the reference posture to generate a multi-dimensional adjustment instruction; the AR-HUD display module is used for calibrating a virtual image based on human eye visual parameters, superposing a three-dimensional arrow direction indication and a semitransparent target body position contour to an actual visual field through a space registration technology, and synchronously displaying a respiratory rhythm guide light band and instruction text content; and the bidirectional feedback module is used for receiving the body position adjustment completion signal of the patient and synchronously displaying an AR view, a skeleton key point matching result and an instruction execution state in real time. According to the invention, the accuracy of posture adjustment of the patient and the treatment efficiency are remarkably improved, and the risk of treatment effect reduction caused by action deviation is effectively reduced.

Description

Background Art

[0002] Currently, there are many pain points in the radiotherapy treatment process. The method relying on the traditional technician's commands has obstacles in language understanding. Especially when facing patients with dialects or hearing impairments, the communication efficiency is significantly reduced. At the same time, the noise interference generated by the operation of equipment in the treatment room further weakens the clarity of voice commands, making it difficult for patients to accurately understand the technician's guidance. In addition, patients are prone to deviations in action execution due to nervousness or lack of immediate feedback, affecting the treatment effect. Existing solutions such as paper diagrams lack dynamic guidance, traditional projection devices cannot be personalized and adapted according to the patient's perspective, and the voice prompt system lacks spatial positioning ability and is difficult to provide accurate spatial guidance.

[0003] Therefore, there is an urgent need to provide a technical solution to solve the above problems. Summary of the Invention

[0004] To solve the above technical problems, the present invention provides a patient pose positioning adaptive guidance system and method.

[0005] In the first aspect, the present invention provides a patient pose positioning adaptive guidance system, including the following technical solutions:

[0006] It includes: a multi-modal perception module, an intelligent processing module, an AR-HUD display module, and a two-way feedback module;

[0007] The multi-modal perception module is used to: when the patient lies on the treatment bed, obtain the patient's skeletal key point sequence, body pressure distribution data, and human eye vision parameters;

[0008] The intelligent processing module is used to: by fusing the skeletal key point sequence, the body pressure distribution data, and the human eye vision parameters, calculate the real-time pose of the patient, and match the patient's personalized reference pose with the real-time pose to generate a multi-dimensional adjustment instruction including direction and amplitude parameters;

[0009] The AR-HUD display module is used to: based on the human eye vision parameters, dynamically calibrate the depth of field parameters and spatial display positions of the virtual image, and superimpose the three-dimensional arrow direction indication and the semi-transparent target body position contour on the patient's actual field of view through spatial registration technology, and synchronously display the breathing rhythm guiding light band and the text content corresponding to the multi-dimensional adjustment instruction;

[0010] The two-way feedback module is used to: receive the body position adjustment completion signal sent by the patient after adjusting according to the multi-dimensional adjustment instruction, and synchronously display the AR view, the skeletal key point matching result, and the instruction execution status in real time.

[0011] Further, the multimodal perception module includes: an infrared depth camera, a distributed pressure sensing array, and a pupil tracking unit; the human eye visual parameters include: line-of-sight focus coordinates and interpupillary distance data;

[0012] The infrared depth camera is deployed on the top and side of the treatment room for real-time acquisition of the sequence of the patient's bone key points;

[0013] The distributed pressure sensing array is integrated on the surface of the treatment bed for detecting the body pressure distribution data of the patient;

[0014] The pupil tracking unit is mounted on the AR-HUD display module through an adjustable bracket for real-time acquisition of the line-of-sight focus coordinates and interpupillary distance data of the patient.

[0015] Further, the intelligent processing module is specifically configured to:

[0016] Generate the personalized reference posture based on the bone structure data and the initial body pressure distribution data scanned by the patient before treatment.

[0017] Further, the intelligent processing module is specifically configured to:

[0018] Call the target language type from the multilingual instruction library based on the preset language preference information of the patient, and generate the multi-dimensional adjustment instruction according to the target language type.

[0019] Further, the AR-HUD display module is specifically configured to:

[0020] Calculate the spatial offset between the real-time posture and the personalized reference posture, and trigger a visual warning signal when the spatial offset is greater than the target threshold.

[0021] Further, the visual warning signal includes: a first visual warning signal and a second visual warning signal; the AR-HUD display module is specifically configured to:

[0022] When the spatial offset is greater than the first threshold and less than the second threshold, trigger the first visual warning signal, and the first visual warning signal is prompted by a yellow flashing border;

[0023] When the spatial offset is greater than the second threshold, trigger the second visual warning signal, and the second visual warning signal is a red mask warning.

[0024] Further, the three-dimensional arrow direction indication is driven and generated by the direction parameter in the multi-dimensional adjustment instruction; the semi-transparent target body position contour is rendered based on the personalized reference posture and compared with the three-dimensional space coordinates of the real-time posture.

[0025] Further, the two-way feedback module is specifically configured to:

[0026] When the patient finishes the adjustment according to the multi-dimensional adjustment instruction, receive the position adjustment completion signal sent by the patient through the handheld button or voice instruction, and synchronously display the AR view, the bone key point matching result, and the instruction execution status in real time through the monitoring interface;

[0027] Among them, the bone key point matching result is: the offset area between the real-time posture and the personalized reference posture presented in the form of a differential heat map; the instruction execution status includes: the instruction sending time, the patient response delay, and the target language type.

[0028] Further, the two-way feedback module is also configured to:

[0029] Output the treatment bed displacement and the patient's autonomous movement trajectory determined based on the fault tolerance compensation mechanism.

[0030] In a second aspect, the present invention provides a patient posture positioning adaptive guidance method, including the following technical solutions:

[0031] When the patient lies on the treatment bed, obtain the patient's bone key point sequence, body pressure distribution data, and human eye vision parameters;

[0032] By fusing the bone key point sequence, the body pressure distribution data, and the human eye vision parameters, calculate the patient's real-time posture, and match the patient's personalized reference posture with the real-time posture to generate a multi-dimensional adjustment instruction including direction and amplitude parameters;

[0033] Based on the human eye vision parameters, dynamically calibrate the depth of field parameters and the spatial display position of the virtual image, and superimpose the three-dimensional arrow direction indication and the semi-transparent target body position contour on the patient's actual field of view through the spatial registration technology, and synchronously display the breathing rhythm guiding light band and the text content corresponding to the multi-dimensional adjustment instruction;

[0034] Receive the position adjustment completion signal sent by the patient after the adjustment according to the multi-dimensional adjustment instruction, and synchronously display the AR view, the bone key point matching result, and the instruction execution status in real time.

[0035] The present invention obtains the patient's skeletal key points, body pressure distribution, and human eye vision parameters in real time through a multi-modal perception module, combines the generation of personalized posture matching and adjustment instructions by an intelligent processing module, uses an AR-HUD display module to dynamically calibrate virtual images and superimpose three-dimensional arrows, target body position contours, and respiratory rhythm guiding light bands, and simultaneously displays the adjustment results in real time through a two-way feedback module. This system solves the problems of traditional voice commands being interfered by noise, low communication efficiency, lack of dynamic guidance, and spatial positioning ability, significantly improves the accuracy and treatment efficiency of patient posture adjustment, is especially suitable for patients with dialect or hearing impairments, and effectively reduces the risk of reduced treatment effects caused by movement deviations.

[0036] Other advantages, objectives, and features of the present invention will be described to some extent in the subsequent description, and to some extent, will be obvious to those skilled in the art based on the study of the following text, or can be taught from the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention, and for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0038] Figure 1 FIG. is a schematic structural diagram of a patient posture positioning adaptive guidance system;

[0039] Figure 2 FIG. is a schematic structural diagram of a multi-modal perception module;

[0040] Figure 3 FIG. is a schematic flow diagram of a patient posture positioning adaptive guidance method. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0041] The following will clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts belong to the scope of protection of the present invention.

[0042] Figure 1 FIG. shows a schematic structural diagram of an embodiment of a patient posture positioning adaptive guidance system 100 provided by the present invention. As Figure 1As shown, the system 100 includes: a multimodal perception module 110, an intelligent processing module 120, an AR-HUD display module 130, and a two-way feedback module 140;

[0043] The multimodal perception module 110 is used to: when the patient lies on the treatment bed, obtain the patient's skeletal key point sequence, body pressure distribution data, and human eye vision parameters.

[0044] Among them, as Figure 2 shown, the multimodal perception module 110 includes: an infrared depth camera 111, a distributed pressure sensing array 112, and a pupil tracking unit 113; the human eye vision parameters include: the line-of-sight focus coordinates and pupil distance data. The infrared depth camera 111 is deployed at the top and side of the treatment room for real-time acquisition of the patient's skeletal key point sequence; the distributed pressure sensing array 112 is integrated on the surface of the treatment bed for detecting the patient's body pressure distribution data; the pupil tracking unit 113 is mounted on the AR-HUD display module 130 through an adjustable bracket for real-time acquisition of the patient's line-of-sight focus coordinates and pupil distance data.

[0045] Among them, the skeletal key point sequence is: a dynamic data stream formed by real-time capturing of the three-dimensional coordinates on the patient's body surface by the infrared depth camera 111 and extracting the joint position marker points based on the human anatomy model and arranging them in chronological order. The set of joint position marker points usually includes 25 core joint points (such as the cervical vertebra, shoulders, elbows, hip joints, knees, ankles, etc.), and each key point contains three-dimensional coordinates (X / Y / Z axis positions); the acquisition frequency of the dynamic data stream is defaulted to 20Hz and can also be adjusted according to actual needs, without limitation here. In this embodiment, by collecting the skeletal key point sequence, it can replace the traditional visual inspection by technicians, provide dynamic body position monitoring with millimeter-level accuracy, and eliminate the positioning deviation caused by the patient's nervous shaking.

[0046] Among them, the body pressure distribution data is: the pressure distribution of the contact area between the patient's body and the bed surface recorded in matrix form by the distributed pressure sensor array 112 integrated on the surface of the treatment bed. The array density of the distributed pressure sensor array 112 is defaulted to: one piezoelectric sensor is arranged every 5cm×5cm to cover the effective area of the treatment bed, the sampling frequency is defaulted to: 10Hz, and the output pressure value range is 0-100kPa (corresponding to the human lying pressure range). In this embodiment, by collecting the body pressure distribution data, the limitations of traditional single skeletal data can be eliminated, and the reliability of posture interpretation can be enhanced through physical contact verification. For example, when the skeletal key points show that the patient has not moved, but the pressure distribution changes suddenly, it is determined that it is the displacement of the treatment bed rather than the patient's movement.

[0047] Among them, the human eye visual parameters are a set of dynamic parameters for the physiological characteristics of the patient's eyes and visual focus information obtained by the pupil tracking unit 113, which are used to adapt the AR display effect. Specifically: the interpupillary distance data is the horizontal distance between the centers of the two pupils of the patient (unit: mm), which is used for the stereoscopic display calibration of the AR image. The line-of-sight focus coordinates are the position of the patient's fixation point in three-dimensional space (X / Y / Z axis coordinates), which are used to dynamically adjust the display position of the AR guidance information. Since traditional projection devices cannot dynamically adjust according to the patient's perspective, the guidance information deviates from the effective visual field. In this embodiment, the human eye visual parameters are collected to realize real-time tracking of eye data, ensuring that the AR information is always in the optimal visible area.

[0048] It should be noted that in this embodiment, the sequence of skeletal key points provides the limb movement trajectory, the body pressure distribution data verifies the body contact stability, and the human eye visual parameters determine the AR display adaptation rules. The pupil tracking unit 113 is integrated in the AR-HUD display module 130 and works when the patient wears a lightweight AR wearable device.

[0049] The intelligent processing module 120 is used to: calculate the real-time posture of the patient by fusing the sequence of skeletal key points, the body pressure distribution data and the human eye visual parameters, and match the personalized reference posture of the patient with the real-time posture to generate a multi-dimensional adjustment instruction including direction and amplitude parameters.

[0050] Among them, the real-time posture refers to the three-dimensional space representation of the patient's real-time body position state during radiotherapy, which is obtained by fusing multi-modal data (sequence of skeletal key points, body pressure distribution data and human eye visual parameters) and is used to dynamically feedback the deviation between the patient's current body position and the reference posture. The personalized reference posture refers to the ideal treatment body position template generated by scanning and filing before treatment, which is used as a reference benchmark for real-time posture adjustment. The multi-dimensional adjustment instruction refers to the spatial pose adjustment instruction generated according to the deviation between the real-time posture and the reference posture, including translation direction, translation amplitude and rotation angle parameters.

[0051] In an optional manner, the intelligent processing module 120 is specifically used to:

[0052] Generate the personalized reference posture based on the skeletal structure data and the initial body pressure distribution data scanned by the patient before treatment.

[0053] Specifically: ① Obtain the skeletal structure data S b scanned by the patient before treatment and the initial body pressure distribution data D b . ② Extract 25 core joint points (anatomical key points) K b from the skeletal structure data S base ={k1,k2,...,k25 , each core joint point k i = {x i , y i , z i}. ③ Perform Gaussian filtering on the initial body pressure distribution data D b to eliminate noise interference and obtain the reference body pressure matrix D base = GaussianFilter(D b , σ = 1.5). ④ Use the Iterative Closest Point (ICP) algorithm to align K base with the treatment bed coordinate system and calculate the transformation matrix representing the position of the i-th core joint point in the treatment bed coordinate system. ⑤ Bind the registered core joint points to the reference body pressure matrix D base to generate the final personalized reference posture P base = (T(K base ), D base ).

[0054] In an alternative approach, the steps of calculating the real-time posture of the patient by the intelligent processing module 120 through fusing the bone key point sequence, body pressure distribution data, and human eye vision parameters specifically include: ① Obtain the bone key point sequence K real , body pressure distribution data D real and the line-of-sight focus coordinates G real = (x g , y g , z g ) in the human eye vision parameters. ② Correct the bone projection error of the bone key point sequence according to the line-of-sight focus coordinates (to avoid the offset of the core joint points caused by the perspective tilt) to obtain the corrected bone key point sequence: K' real = K real ·R(α, β); R(α, β) represents the rotation matrix that rotates α degrees around the X-axis and β degrees around the Y-axis, which can be calculated from the visual focus coordinates. ③ Calculate the body pressure distribution similarity according to the body pressure distribution data D real and the reference body pressure matrix D base If SIM < 0.9, it is determined that the body pressure is abnormal (such as the patient moving). ④ If SIM ≥ 0.9, weighted fusion of the corrected bone key point sequence K' real and the body pressure distribution data D real to generate the real-time posture P real = w k ·K' real + w d ·Normalize(D real ​)); where w k and w d are weight coefficients, which can be specifically calibrated according to clinical data and are not restricted here. For example, in this embodiment, w k = 0.7, w d = 0.3.

[0055] In an alternative manner, the step of matching the patient's personalized reference posture with the real-time posture in the intelligent processing module 120 to generate a multi-dimensional adjustment instruction including direction and amplitude parameters specifically includes: ① Using the ICP algorithm, align the personalized reference posture with the real-time posture, calculate the rotation matrix R and the translation vector t, calculate the rotation angle θ according to the rotation matrix R, and determine the translation vector t as the preliminary bone offset ΔP 骨骼 . ② When the body pressure distribution similarity SIM ≥ 0.9, execute step ③; when the body pressure distribution similarity SIM < 0.9, the bone data may be interfered by the displacement of the treatment bed and needs to be corrected in combination with the line of sight focus. ③ According to the line of sight focus coordinates G real correct the projection error of the step bone offset ΔP 骨骼 to obtain the bone correction amount G base represents the line of sight focus coordinates under the reference posture. ④ Generate a multi-dimensional adjustment instruction

[0056] In an alternative manner, the intelligent processing module 120 is specifically configured to:

[0057] Based on the preset language preference information of the patient, call the target language type from the multi-language instruction library and generate the multi-dimensional adjustment instruction according to the target language type.

[0058] Specifically: convert the multi-dimensional adjustment instruction into a natural language description; call the target language type from the multi-language instruction library according to the preset language preference information and generate the multi-dimensional adjustment instruction according to the target language type.

[0059] The AR-HUD display module 130 is configured to: dynamically calibrate the depth of field parameters and the spatial display position of the virtual image based on the human eye vision parameters, superimpose the three-dimensional arrow direction indication and the semi-transparent target body position contour on the actual field of view of the patient through the spatial registration technology, and synchronously display the breathing rhythm guiding light band and the text content corresponding to the multi-dimensional adjustment instruction.

[0060] Among them, the three-dimensional arrow direction indication is driven and generated by the direction parameter in the multi-dimensional adjustment instruction; the semi-transparent target body position contour is rendered based on the personalized reference posture and compared with the three-dimensional space coordinates of the real-time posture.

[0061] In an alternative approach, the steps of dynamically calibrating the depth of field parameter and the spatial display position of the virtual image based on the human eye vision parameters in the AR-HUD display module 130 specifically include: ① Calculate the depth of field correction amount Δd = k d ·(IPD - IPD0); IPD0 is the standard pupil distance, defaulting to 64 mm; IPD is the actually collected pupil distance data, and k d is the depth of field adaptation coefficient, which is calibrated according to the optical system, and k d defaults to 0.5 mm / mm. ② Dynamically calibrate the depth of field parameter d new = d0 + Δd according to the initial depth of field d0 and the depth of field correction amount Δd in the AR display parameters. ③ Calculate the spatial display position offset vector ΔP = G real - G base ; G base = (0, 0, 1.5) m, and dynamically calibrate the spatial display position The calibrated AR display parameters (depth of field parameter and spatial display position) are (d new , P new ).

[0062] In an alternative approach, the steps of superimposing the three-dimensional arrow direction indication and the translucent target body position contour onto the actual field of view of the patient through the spatial registration technology in the AR-HUD display module 130 specifically include: ① Determine the direction vector v dir of the three-dimensional arrow direction indication, the reference pose P base corresponding to the translucent target body position contour, the skeletal key points K base and the transformation matrix M view from the real scene to the AR scene constructed in real time by the infrared depth camera 111. ② Project the direction vector onto the patient's field of view plane v proj = M view ·v dir ; Generate the arrow tip coordinates: ArrowMesh = GenerateArrow(v proj , Δd). ③ Convert the skeletal key points K base to the patient's field of view coordinates K view = M view ·K base ; Generate the translucent contour ContourMesh = CreateContour(K view , α = 0.3). ④ Superimpose ArrowMesh and ContourMesh onto the AR-HUD display module 130 through the rendering engine:

[0063] In an alternative manner, the steps of synchronously displaying the breathing rhythm guiding light band and the text content corresponding to the multi-dimensional adjustment instruction in the AR-HUD display module 130 specifically include: adjusting the light band length according to the breathing phase based on the thoracic displacement waveform collected in real time by the respiratory gating device; performing spatio-temporal alignment on the text content; and outputting and displaying the light band and the overlay on the same AR layer.

[0064] In an alternative manner, the AR-HUD display module 130 is specifically configured to:

[0065] Calculate the spatial offset between the real-time pose and the personalized reference pose, and trigger a visual warning signal when the spatial offset is greater than the target threshold.

[0066] Wherein, the spatial offset is obtained by calculating the average offset of the skeletal key points and the body pressure distribution offset between the real-time pose and the personalized reference pose, and performing weighted fusion.

[0067] Wherein, the target threshold includes: a first threshold and a second threshold. The visual warning signal includes: a first visual warning signal and a second visual warning signal.

[0068] It should be noted that the first threshold is defaulted to 3mm, and the second threshold is defaulted to 5mm. They can also be adjusted according to the actual situation, as long as the first threshold is less than the second threshold. The visual warning signal can also be adjusted according to actual needs, and there is no limitation here. Specifically; when the spatial offset is greater than the first threshold and less than the second threshold, the first visual warning signal is triggered, and the first visual warning signal uses a yellow flashing border for prompt; when the spatial offset is greater than the second threshold, the second visual warning signal is triggered, and the second visual warning signal uses a red mask for warning. When the spatial offset is less than the first threshold, no visual warning signal is issued.

[0069] The two-way feedback module 140 is configured to: receive the body position adjustment completion signal sent by the patient after adjusting according to the multi-dimensional adjustment instruction, and synchronously display the AR view, the skeletal key point matching result, and the instruction execution status in real time.

[0070] Specifically, when the patient completes the adjustment according to the multi-dimensional adjustment instruction, receive the body position adjustment completion signal sent by the patient through the handheld button or voice instruction, and synchronously display the AR view, the skeletal key point matching result, and the instruction execution status in real time through the monitoring interface;

[0071] Wherein, the skeletal key point matching result is: the offset area between the real-time pose and the personalized reference pose presented in the form of a difference heat map; the instruction execution status includes: the instruction sending time, the patient response delay, and the target language type.

[0072] In an alternative manner, the bidirectional feedback module 140 is further configured to:

[0073] Output the displacement of the treatment bed and the patient's autonomous movement trajectory determined based on the fault tolerance compensation mechanism.

[0074] Among them, the displacement of the treatment bed and the patient's autonomous movement trajectory can be distinguished by different color trajectory lines on the monitoring interface, and key indicators (such as frequency ratio, direction consistency value, etc.) for determining the movement are displayed.

[0075] The technical solution of this embodiment obtains the patient's skeletal key points, body pressure distribution, and human eye visual parameters in real time through the multi-modal perception module, combines the generation of personalized posture matching and adjustment instructions by the intelligent processing module, uses the AR-HUD display module to dynamically calibrate the virtual image and superimpose three-dimensional arrows, target body position contours, and breathing rhythm guiding light bands, and simultaneously displays the adjustment results in real time through the bidirectional feedback module. This embodiment solves the problems of traditional voice commands being interfered by noise, low communication efficiency, lack of dynamic guidance and spatial positioning capabilities, significantly improves the accuracy and treatment efficiency of patient posture adjustment, is especially suitable for patients with dialects or hearing impairments, and effectively reduces the risk of reduced treatment effects caused by movement deviations.

[0076] Figure 3 The flowchart of an embodiment of a patient posture positioning adaptive guidance method provided by the present invention is shown. As Figure 3 shown, the method includes the following steps:

[0077] S1. When the patient lies on the treatment bed, obtain the patient's sequence of skeletal key points, body pressure distribution data, and human eye visual parameters.

[0078] S2. By fusing the sequence of skeletal key points, the body pressure distribution data, and the human eye visual parameters, calculate the real-time posture of the patient, and match the patient's personalized reference posture with the real-time posture to generate multi-dimensional adjustment instructions including direction and amplitude parameters.

[0079] S3. Based on the human eye visual parameters, dynamically calibrate the depth of field parameters and spatial display position of the virtual image, and superimpose the three-dimensional arrow direction indication and semi-transparent target body position contour on the patient's actual field of view through spatial registration technology, and simultaneously display the breathing rhythm guiding light band and the text content corresponding to the multi-dimensional adjustment instructions.

[0080] S4. Receive the body position adjustment completion signal sent by the patient after adjusting according to the multi-dimensional adjustment instructions, and simultaneously display the AR view, the skeletal key point matching result, and the instruction execution status in real time.

[0081] In an alternative manner, the multi-modal perception module includes: an infrared depth camera, a distributed pressure sensing array, and a pupil tracking unit; the human eye visual parameters include: line-of-sight focus coordinates and interpupillary distance data;

[0082] The infrared depth camera is deployed at the top and sides of the treatment room for real-time acquisition of the sequence of the patient's bone key points;

[0083] The distributed pressure sensing array is integrated on the surface of the treatment bed for detecting the body pressure distribution data of the patient;

[0084] The pupil tracking unit is mounted on the AR-HUD display module through an adjustable bracket for real-time acquisition of the line-of-sight focus coordinates and interpupillary distance data of the patient.

[0085] In an alternative manner, the steps of obtaining the patient's personalized reference posture include:

[0086] Generating the personalized reference posture based on the bone structure data and the initial body pressure distribution data scanned before treatment of the patient.

[0087] In an alternative manner, the steps of generating a multi-dimensional adjustment instruction including direction and amplitude parameters include:

[0088] Based on the preset language preference information of the patient, calling the target language type from the multi-language instruction library and generating the multi-dimensional adjustment instruction according to the target language type.

[0089] In an alternative manner, it further includes:

[0090] Calculating the spatial offset between the real-time posture and the personalized reference posture, and triggering a visual warning signal when the spatial offset is greater than a target threshold.

[0091] In an alternative manner, the visual warning signal includes: a first visual warning signal and a second visual warning signal; the steps of triggering the visual warning signal when the spatial offset is greater than the target threshold include:

[0092] When the spatial offset is greater than a first threshold and less than a second threshold, triggering the first visual warning signal, and the first visual warning signal is prompted by a yellow flashing border;

[0093] When the spatial offset is greater than the second threshold, triggering the second visual warning signal, and the second visual warning signal is a red mask warning.

[0094] In an alternative manner, the three-dimensional arrow direction indication is driven and generated by the direction parameter in the multi-dimensional adjustment instruction; the semi-transparent target body position contour is rendered based on the personalized reference posture and compared with the three-dimensional space coordinates of the real-time posture.

[0095] In an alternative manner, S4 includes:

[0096] When the patient finishes adjusting according to the multi-dimensional adjustment instruction, receive the posture adjustment completion signal sent by the patient through the handheld button or voice instruction, and synchronously display the AR view, the bone key point matching result, and the instruction execution status in real time through the monitoring interface;

[0097] Wherein, the bone key point matching result is the offset area between the real-time posture and the personalized reference posture presented in the form of a differential heat map; the instruction execution status includes: the instruction sending time, the patient response delay, and the target language type.

[0098] In an alternative manner, it further includes:

[0099] Output the treatment bed displacement and the patient's autonomous movement trajectory determined based on the fault tolerance compensation mechanism.

[0100] The technical solution of this embodiment obtains the patient's bone key points, body pressure distribution, and human eye vision parameters in real time through the multi-modal perception module, combines the personalized posture matching and adjustment instruction generation of the intelligent processing module, uses the AR-HUD display module to dynamically calibrate the virtual image and superimpose the three-dimensional arrow, the target body position contour, and the breathing rhythm guiding light band, and at the same time synchronously displays the adjustment result in real time through the two-way feedback module. This embodiment solves the problems of traditional voice instructions being interfered by noise, low communication efficiency, lack of dynamic guidance and spatial positioning ability, significantly improves the accuracy of patient posture adjustment and the treatment efficiency, is especially suitable for patients with dialects or hearing impairments, and effectively reduces the risk of treatment effect decline caused by movement deviation.

[0101] An electronic device according to an embodiment of the present invention includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements any one of the above-mentioned patient posture positioning adaptive guidance methods. That is to say, an electronic device according to an embodiment of the present invention may include, but is not limited to: a processor and a memory; the memory is used to store the computer program; the processor is used to execute the patient posture positioning adaptive guidance method shown in any one of the embodiments of the present invention by calling the computer program.

[0102] A computer-readable storage medium according to an embodiment of the present invention stores a computer program thereon, and when the computer program is executed by a processor, the above-mentioned any one of the patient posture positioning adaptive guidance methods is implemented.

[0103] Optionally, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, floppy disk, and optical data storage device, etc.

[0104] In an exemplary embodiment, there is also provided a computer program product or a computer program, the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device executes the above-mentioned patient posture positioning adaptive guidance method.

[0105] It should be understood that the flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of the methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order from that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, as well as the combination of blocks in the block diagram and / or flowchart, may be implemented by a dedicated hardware-based system for performing the specified functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions.

[0106] The computer-readable storage medium provided by the embodiments of the present invention may be, but is not limited to, a system, device or component of electricity, magnetism, light, electromagnetic, infrared ray, or semiconductor, or any combination of the above. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, the computer-readable storage medium may be any tangible medium that contains or stores a program, and this program may be used by or in combination with an instruction execution system, device or component.

[0107] The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed by the electronic device, the electronic device is caused to execute the method shown in the above embodiments.

[0108] The above description is only a preferred embodiment of the present invention and an explanation of the applied technical principles. Those skilled in the art should understand that the scope of disclosure involved in the present invention is not limited to the technical solution formed by the specific combination of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosure concept. For example, the technical solution formed by mutually replacing the above features with the technical features (but not limited to) having similar functions disclosed in the present invention.

[0109] It should be noted that the terms "first", "second", etc. in the description and claims of this application are used to distinguish similar objects, and are used to limit a specific order or sequence. The order of use of similar objects may be interchanged appropriately, so that the embodiments of this application described here can be implemented in an order other than the order shown or described.

[0110] Those skilled in the art know that the present invention can be implemented as a system, method or computer program product. Therefore, the present invention can be specifically implemented in the following forms: it can be completely hardware, can also be completely software (including firmware, resident software, microcode, etc.), and can also be in the form of a combination of hardware and software, which is generally referred to as "circuit", "module" or "system" in this article. In addition, in some embodiments, the present invention can also be implemented in the form of a computer program product in one or more computer-readable media, and the computer-readable media contains computer-readable program code.

[0111] Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention.

Claims

1. An adaptive guidance system based on patient pose positioning, characterized in that Including: A multi-modal perception module, an intelligent processing module, an AR-HUD display module, and a two-way feedback module; The multi-modal perception module is used to: when the patient lies on the treatment bed, obtain the patient's skeletal key point sequence, body pressure distribution data, and human eye vision parameters; The intelligent processing module is used to: by fusing the skeletal key point sequence, the body pressure distribution data, and the human eye vision parameters, calculate the real-time posture of the patient, and match the patient's personalized reference posture with the real-time posture to generate a multi-dimensional adjustment instruction including direction and amplitude parameters; The AR-HUD display module is used to: based on the human eye vision parameters, dynamically calibrate the depth of field parameters and spatial display position of the virtual image, and overlay the three-dimensional arrow direction indication and the semi-transparent target body position contour onto the actual field of view of the patient through spatial registration technology, and synchronously display the breathing rhythm guiding light band and the text content corresponding to the multi-dimensional adjustment instruction; The two-way feedback module is used to: receive the body position adjustment completion signal sent by the patient after adjusting according to the multi-dimensional adjustment instruction, and synchronously display the AR view, the skeletal key point matching result, and the instruction execution status in real time.

2. The adaptive guidance system for patient posture positioning according to claim 1, wherein, The multi-modal perception module includes: an infrared depth camera, a distributed pressure sensing array, and a pupil tracking unit; the human eye vision parameters include: line-of-sight focus coordinates and pupil distance data; The infrared depth camera is deployed on the top and side of the treatment room for real-time collection of the patient's skeletal key point sequence; The distributed pressure sensing array is integrated on the surface of the treatment bed for detecting the patient's body pressure distribution data; The pupil tracking unit is mounted on the AR-HUD display module through an adjustable bracket for real-time acquisition of the patient's line-of-sight focus coordinates and pupil distance data.

3. The adaptive guidance system for patient posture positioning according to claim 1, characterized in that The intelligent processing module is specifically used to: Generate the personalized reference posture based on the skeletal structure data and the initial body pressure distribution data scanned by the patient before treatment.

4. The adaptive guidance system based on patient posture positioning according to claim 1 or 3, characterized in that, The intelligent processing module is specifically used to: Based on the patient's preset language preference information, call the target language type from the multi-language instruction library, and generate the multi-dimensional adjustment instruction according to the target language type.

5. The adaptive guidance system for patient posture positioning according to claim 1, wherein, The AR-HUD display module is specifically used to: Calculate the spatial offset between the real-time posture and the personalized reference posture, and trigger a visual warning signal when the spatial offset is greater than the target threshold.

6. The adaptive guidance system for patient posture positioning according to claim 5, characterized in that The visual warning signal includes: a first visual warning signal and a second visual warning signal; the AR-HUD display module is specifically used to: When the spatial offset is greater than the first threshold and less than the second threshold, trigger the first visual warning signal, and the first visual warning signal is prompted by a yellow flashing border; When the spatial offset is greater than the second threshold, trigger the second visual warning signal, and the second visual warning signal is warned by a red mask.

7. The adaptive guidance system for patient posture positioning according to claim 1, wherein The three-dimensional arrow direction indication is driven and generated by the direction parameter in the multi-dimensional adjustment instruction; the semi-transparent target body position contour is rendered based on the personalized reference posture and compared with the three-dimensional space coordinates of the real-time posture.

8. The adaptive guidance system for patient pose positioning according to claim 1, characterized in that, The two-way feedback module is specifically configured to: When the patient finishes adjusting according to the multi-dimensional adjustment instruction, receive the posture adjustment completion signal sent by the patient through the handheld button or voice instruction, and synchronously display the AR view, the bone key point matching result, and the instruction execution status in real time through the monitoring interface; Among them, the bone key point matching result is the offset area between the real-time posture and the personalized reference posture presented in the form of a difference heat map; the instruction execution status includes: the instruction sending time, the patient response delay, and the target language type.

9. The adaptive guidance system for patient posture positioning according to claim 8, wherein The two-way feedback module is further configured to: Output the treatment bed displacement and the patient's autonomous movement trajectory determined based on the fault tolerance compensation mechanism.

10. An adaptive guidance method based on patient pose positioning, characterized in that, Including: When the patient lies on the treatment bed, obtain the patient's bone key point sequence, body pressure distribution data, and human eye vision parameters; By fusing the bone key point sequence, the body pressure distribution data, and the human eye vision parameters, calculate the real-time posture of the patient, match the personalized reference posture of the patient with the real-time posture, and generate a multi-dimensional adjustment instruction including direction and amplitude parameters; Based on the human eye vision parameters, dynamically calibrate the depth of field parameters and the spatial display position of the virtual image, and overlay the three-dimensional arrow direction indication and the semi-transparent target body position contour onto the actual field of view of the patient through the spatial registration technology, and synchronously display the breathing rhythm guiding light band and the text content corresponding to the multi-dimensional adjustment instruction; Receive the posture adjustment completion signal sent by the patient after adjusting according to the multi-dimensional adjustment instruction, and synchronously display the AR view, the bone key point matching result, and the instruction execution status in real time.

Images