Multimodal adaptive positioning system for the left upper limb of patients during coronary intervention

Abstract

The present invention relates to the technical field of medical assistive devices, and more specifically to a multimodal adaptive positioning system for the left upper limb of a patient during coronary intervention. The system comprises: an acquisition module for acquiring surgical scene images and patient vital sign data; a processing module for determining the initial positioning position of the patient's left upper limb; determining a surgical operation period and a corresponding surgical operation space area, and determining the degree of intraoperative interference based on overlapping pixels of the upper limb support plate area and the surgical operation space area in each surgical scene image within the surgical operation period and the operator's position movement factor; determining the patient's upper limb compression index within the surgical operation period; determining a target adjustment value based on the intraoperative interference value and the upper limb compression index; and a positioning module for inputting the target adjustment value into a multi-axis electric push rod support controller to control the multi-axis electric push rod support to adjust the upper limb support plate area to the target position. The present invention can dynamically adjust the support plate position during coronary intervention.

Description

Technical Field

[0001] The present invention relates to the technical field of medical auxiliary equipment, and in particular to a multimodal adaptive positioning system for the left upper limb of a patient during coronary intervention. Background Art

[0002] Coronary artery intervention is a minimally invasive interventional treatment method that involves inserting a catheter into the coronary artery through the blood vessel lumen via skin puncture to dilate, open, and insert a stent in the narrowed or occluded blood vessels to restore myocardial blood supply. When accessing the left radial artery, the patient's left upper limb needs to be fixed on a specific support device to facilitate the doctor's operation on the patient's right side. During coronary intervention surgery, the proper positioning of the patient's left upper limb is crucial to the smooth progress of the operation.

[0003] In the related technology, during the above-mentioned coronary artery intervention treatment, the fixation method for the patient's left upper limb is single, and the fixation position is usually difficult to adjust. Maintaining the posture for a long time can easily cause numbness, soreness or tenderness in the patient's left upper limb, increasing the risk of intraoperative discomfort or postoperative complications. Summary of the Invention

[0004] The present invention provides a multimodal adaptive positioning system for the patient's left upper limb during coronary intervention, so as to solve the problem in the prior art that during coronary intervention, the patient's left upper limb is fixed in a single way and the fixed position is usually difficult to adjust. Maintaining the position for a long time can easily lead to numbness, soreness or tenderness in the patient's left upper limb, increasing the risk of intraoperative discomfort or postoperative complications.

[0005] The multimodal adaptive positioning system for the left upper limb of a patient during coronary intervention of the present invention adopts the following technical solutions:

[0006] One embodiment of the present invention provides a multimodal adaptive positioning system for the left upper limb of a patient during coronary intervention, the system comprising the following modules:

[0007] An acquisition module is used to acquire surgical scene images and patient vital sign data; the surgical scene images cover the surgeon, patient, operating table area, and upper limb support plate area;

[0008] A processing module is used to determine the initial placement position of the patient's left upper limb on the upper limb support plate; and, based on the surgical scene image, determine the surgical operation period and its corresponding surgical operation space area, and based on the overlapping pixel points of the upper limb support plate area and the surgical operation space area in each of the surgical scene images within the surgical operation period and the position movement factor of the operator, determine the interference degree of the position of the upper limb support plate area on the operator's intraoperative operation, the position movement factor including a horizontal movement factor and a longitudinal movement factor, the horizontal movement factor being used to characterize the lateral position change of the operator, and the longitudinal movement factor being used to characterize the longitudinal position change of the operator; determine the upper limb compression index of the patient within the surgical operation period based on the characteristic data; determine the target adjustment value based on the intraoperative operation interference degree and the upper limb compression index;

[0009] The positioning module is used to input the target adjustment value into the multi-axis electric push rod support controller, and use the multi-axis electric push rod support controller to control the multi-axis electric push rod support to adjust the upper limb support plate area to the target position.

[0010] Exemplarily, the vital sign data includes the patient's height data and weight data; the processing module implements the determination of the initial placement position of the patient's left upper limb on the upper limb support tray based on the vital sign data through the following method, including: determining the patient's body mass index based on the height data and the weight data; obtaining the target angle range of the patient's left upper limb abduction, and the target index range consisting of the normal range and the overweight range of the body mass index, and determining the corresponding coefficient of the target angle range and the target index range; determining the initial placement angle based on the patient's body mass index and the corresponding coefficient; obtaining the support fulcrum of the multi-axis electric push rod bracket for the upper limb support tray area, and the proximal end point of the upper limb support tray area closest to the operating table area, and determining the distance between the support fulcrum and the proximal end point; determining the initial placement position based on the initial placement angle and the distance between the support fulcrum and the proximal end point.

[0011] Exemplarily, the processing module implements the determination of the surgical operation time period and its corresponding surgical operation space area based on the surgical scene image through the following method, including: obtaining an image sequence of the surgical scene images collected within a target historical time period; identifying the operator area corresponding to the operator, the patient area corresponding to the patient, the upper limb support plate area and the instrument area in each of the surgical scene images in the image sequence based on image detection and semantic segmentation algorithms; for each frame of the surgical scene image in the image sequence, determining the basic surgical operation space area based on the position information of the operator area and the instrument area in the surgical scene image; for each frame of the surgical scene image in the image sequence, determining the corresponding surgical operation time period based on the upper limb skeleton key points of the operator in the current surgical scene image frame and the next surgical scene image frame, as well as the upper limb skeleton key points of the operator and the wrist key points of the patient in the current surgical scene image frame; determining the surgical operation space area corresponding to the surgical operation time period based on the surgical operation time period and the basic surgical operation space area.

[0012] Exemplarily, the processing module implements the determination of the basic area of ​​the surgical operation space for each frame of the surgical scene image in the image sequence based on the position information of the operator area and the instrument area in the surgical scene image through the following method, including: for each frame of the surgical scene image in the image sequence, determining the instrument area adjacent to the operator area in the current surgical scene image frame, calculating the average value of the difference in depth values ​​of adjacent pixel points between the operator area and the adjacent instrument area; determining the current operating instrument area based on the average value of the difference in depth values ​​of the adjacent pixel points, merging the operator area and the current operating instrument area to obtain the basic area of ​​the surgical operation space.

[0013] Exemplarily, the processing module implements the surgical scene image for each frame in the image sequence by the following method, and determines the corresponding surgical operation period based on the upper limb skeleton key points of the surgeon in the current surgical scene image frame and the next surgical scene image frame, as well as the upper limb skeleton key points of the surgeon in the current surgical scene image frame and the wrist key points of the patient, including: obtaining a first shoulder line between the surgeon's shoulder key point in the current surgical scene image frame and the surgeon's shoulder key point in the next surgical scene image frame, and a second shoulder line between the surgeon's shoulder key point in the current surgical scene image frame and the patient's wrist key point, and determining a first angle between the first shoulder line and the second shoulder line; obtaining a first wrist line between the surgeon's wrist key point in the current surgical scene image frame and the surgeon's wrist key point in the next surgical scene image frame. A connection line is obtained, and a second wrist connection line between the surgeon's wrist key point and the patient's wrist key point in the current surgical scene image frame, and a second wrist connection line is obtained, and a second angle between the first wrist connection line and the second wrist connection line is determined; a first elbow connection line is obtained between the surgeon's elbow key point in the current surgical scene image frame and the surgeon's elbow key point in the next surgical scene image frame, and a second elbow connection line is obtained between the surgeon's elbow key point and the patient's wrist key point in the current surgical scene image frame, and a third angle between the first elbow connection line and the second elbow connection line is determined; an average value between the first angle, the second angle and the third angle is calculated as a surgical operation path evaluation coefficient; the surgical operation path evaluation coefficients of the surgical scene images of each frame in the image sequence are sorted in order of size, and the surgical operation time period is determined based on the obtained surgical operation path evaluation coefficient sequence.

[0014] Exemplarily, the processing module implements the determination of the surgical operation space area corresponding to the surgical operation period based on the surgical operation period and the surgical operation space basic area through the following method, including: for each surgical operation period, obtaining the frame difference pixel points of the surgical operation space basic area between the current surgical scene image frame and the next surgical scene image frame in the surgical operation period, and using the convex hull formed based on the frame difference pixel points as the surgical operation space area corresponding to the surgical operation period.

[0015] Exemplarily, the processing module implements the determination of the interference degree of the position of the upper limb support plate area on the operator's intraoperative operation based on the overlapping pixel points of the upper limb support plate area and the surgical operation space area in each surgical scene image within the surgical operation period and the position movement factor of the operator through the following method, including: for each frame of the surgical scene image within the surgical operation period, obtaining the overlapping pixel points of the upper limb support plate area and the surgical operation space area in the surgical scene image, and recording the position corresponding to the overlapping pixel point as the potential interference position; in other surgical scene image frames except the current surgical scene image frame within the surgical operation period, obtaining the pixel point at the potential interference position is no longer the target position of the overlapping pixel point, and calculating the position of the other surgical scene image frames in the surgical scene image frame. The depth value average of the pixel points at the target position is obtained, and the difference between the depth value average and the depth value of the pixel points at the potential interference position is calculated to obtain the interference error corresponding to the potential interference position; the average value of the interference error corresponding to each potential interference position is calculated to obtain the depth difference factor corresponding to the current surgical scene image frame; for each frame of the surgical scene image within the surgical operation period, the horizontal movement factor and the vertical movement factor of the surgeon corresponding to the current surgical scene image frame are determined based on the position information and depth information of the surgeon's upper limb skeleton key points in the current surgical scene image frame and the next surgical scene image frame; the intraoperative operation interference degree corresponding to the surgical operation period is determined based on the horizontal movement factor, the vertical movement factor and the depth difference factor.

[0016] Exemplarily, the processing module implements the determination of the upper limb compression index of the patient during the surgical operation period based on the characteristic data through the following method, including: obtaining the pressure data of the patient at each time point during the surgical operation period through each pressure sensor on the upper limb support plate area and averaging it; obtaining the blood perfusion index of the patient at each time point during the surgical operation period through the photoelectric volumetric pulse wave sensor on the upper limb support plate area; at each time point during the surgical operation period, calculating the ratio of the pressure mean and the blood perfusion index to obtain the upper limb compression index.

[0017] Exemplarily, the processing module implements the determination of the target adjustment value based on the intraoperative operation interference degree and the upper limb compression index through the following method, including: when the average value of the intraoperative operation interference degree in each surgical operation period is greater than a preset threshold, or the upper limb compression index meets the preset conditions, based on the adjustable direction of the multi-axis electric push rod bracket, and the distance between the patient's wrist key point and the surgeon's shoulder key point, determining the target adjustment direction of the upper limb support plate area; using the intraoperative operation interference degree or the upper limb compression index as an adjustment coefficient, and determining the target adjustment value based on the adjustment coefficient and the preset adjustable range of the upper limb support plate area.

[0018] Exemplarily, the positioning module implements the adjustment of the upper limb support plate area to the target position by controlling the multi-axis electric push rod bracket controller through the following method, including: controlling the multi-axis electric push rod bracket along the target adjustment direction through the multi-axis electric push rod bracket controller to move the upper limb support plate area to the target position corresponding to the target adjustment value.

[0019] The beneficial effects of the technical solution of the present invention are as follows:

[0020] In the multimodal adaptive positioning system for the left upper limb of a patient during coronary intervention provided by the present invention, an acquisition module is used to acquire surgical scene images and vital sign data of the patient; the surgical scene images cover the surgeon, the patient, the operating table area and the upper limb support plate area; a processing module is used to determine the initial placement position of the patient's left upper limb on the upper limb support plate; and, based on the surgical scene images, a surgical operation period and its corresponding surgical operation space area are determined, based on the overlapping pixel points of the upper limb support plate area and the surgical operation space area in each surgical scene image within the surgical operation period and the surgeon's position movement factor, Determine the degree of interference of the position of the upper limb support plate area on the surgeon's intraoperative operation. The above-mentioned position movement factor includes a horizontal movement factor and a longitudinal movement factor. The horizontal movement factor is used to characterize the surgeon's lateral position change, and the longitudinal movement factor is used to characterize the surgeon's longitudinal position change; determine the patient's upper limb compression index during the surgical operation period based on the characteristic data; determine the target adjustment value based on the intraoperative operation interference degree and the upper limb compression index; a positioning module is used to input the target adjustment value into the multi-axis electric push rod bracket controller to control the multi-axis electric push rod bracket through the multi-axis electric push rod bracket controller to adjust the upper limb support plate area to the target position. The present invention collects surgical scene images and patient vital sign data through the above-mentioned acquisition module, so that the processing module can determine the initial placement position of the patient's left upper limb based on the collected patient vital sign data, and then adjust the upper limb support plate area to the initial placement position through the multi-axis electric push rod bracket. The patient's characteristic data is taken into account in the process of determining the initial placement position, and a suitable initial placement position can be determined for different patients, which helps to alleviate the patient's intraoperative discomfort; in addition, during the operation, the present invention determines the position of the upper limb support plate area in each surgical operation period based on the collected surgical scene images and the patient's vital sign data through the above-mentioned processing module. The target adjustment value of the upper limb support plate area is determined based on the intraoperative operation interference degree and the upper limb compression index, and the position of the above-mentioned upper limb support plate area is dynamically adjusted based on the determined target adjustment value through the multi-axis electric push rod bracket, so that the discomfort of the patient's surgical side limb during coronary intervention can be reduced on the premise of facilitating the doctor's intraoperative operation, thereby improving the operational efficiency of coronary intervention. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0022] Figure 1This is a structural block diagram of the multimodal adaptive positioning system for the left upper limb of a patient during coronary intervention provided by the present invention;

[0023] Figure 2 This is a schematic diagram of the upper limb support plate in the adaptive positioning system for the left upper limb of a patient during multimodal coronary intervention provided by the present invention. DETAILED DESCRIPTION

[0024] To further illustrate the technical means and effectiveness of the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, describes in detail the specific implementation, structure, features, and effectiveness of the multimodal adaptive positioning system for the left upper limb of a patient during percutaneous coronary intervention proposed by the present invention. In the following description, different references to "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics of one or more embodiments may be combined in any suitable manner.

[0025] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0026] The specific scheme of the adaptive positioning system for the left upper limb of a patient during coronary intervention based on multimodality provided by the present invention is described in detail below with reference to the accompanying drawings.

[0027] See also Figure 1 , which shows a structural block diagram of a multimodal adaptive positioning system for the left upper limb of a patient during coronary intervention, provided by one embodiment of the present invention. The system includes the following modules:

[0028] The acquisition module 110 is used to acquire surgical scene images and patient vital sign data; the surgical scene images cover the surgeon, patient, operating table area, and upper limb support plate area;

[0029] The processing module 120 is configured to determine the initial placement position of the patient's left upper limb on the upper limb support plate based on the vital sign data; determine the surgical operation period and its corresponding surgical operation space area based on the surgical scene image; determine the degree of interference of the position of the upper limb support plate area with the surgeon's intraoperative operation based on the overlapping pixel points of the upper limb support plate area and the surgical operation space area in each surgical scene image within the surgical operation period and the surgeon's position movement factor, wherein the above-mentioned position movement factor includes a horizontal movement factor and a vertical movement factor, the horizontal movement factor is used to characterize the surgeon's lateral position change, and the vertical movement factor is used to characterize the surgeon's longitudinal position change; determine the patient's upper limb compression index within the surgical operation period based on the characteristic data; and determine a target adjustment value based on the intraoperative operation interference degree and the upper limb compression index;

[0030] The positioning module 130 is used to input the target adjustment value into the multi-axis electric push rod support controller so as to control the multi-axis electric push rod support to adjust the upper limb support plate area to the target position through the multi-axis electric push rod support controller.

[0031] The present invention collects surgical scene images and patient vital sign data through the above-mentioned acquisition module, so that the processing module can determine the initial placement position of the patient's left upper limb based on the collected patient vital sign data, and then adjust the upper limb support plate area to the initial placement position through the multi-axis electric push rod bracket. The patient's characteristic data is taken into account in the process of determining the initial placement position, and a suitable initial placement position can be determined for different patients, which helps to alleviate the patient's intraoperative discomfort; in addition, during the operation, the present invention determines the position of the upper limb support plate area in each surgical operation period based on the collected surgical scene images and the patient's vital sign data through the above-mentioned processing module. The target adjustment value of the upper limb support plate area is determined based on the intraoperative operation interference degree and the upper limb compression index, and the position of the above-mentioned upper limb support plate area is dynamically adjusted based on the determined target adjustment value through the multi-axis electric push rod bracket, so that the discomfort of the patient's surgical side limb during coronary intervention can be reduced on the premise of facilitating the doctor's intraoperative operation, thereby improving the operational efficiency of coronary intervention.

[0032] The multimodal, adaptive positioning system for the left upper limb of a patient during coronary intervention provided in an embodiment of the present application is used to dynamically adjust the position of the left upper limb of a patient during coronary intervention. Below, in one embodiment, the functions of the various modules of the multimodal, adaptive positioning system for the left upper limb of a patient during coronary intervention are described in detail:

[0033] The acquisition module 110 is used to acquire surgical scene images and patient vital sign data; the surgical scene images cover the surgeon, patient, operating table area and upper limb support plate area.

[0034] In an embodiment of the present application, the above-mentioned surgical scene image is an image of the surgical process captured in real time by an image acquisition device. The surgical scene image needs to cover the surgeon, patient, operating table area and upper limb support plate area. Exemplarily, a top-view red-green-blue depth (RGB-D) camera can be set in the above-mentioned acquisition module, and the surgical scene image during the operation is captured in real time by the RGB-D camera. The surgical scene image captured by the RGB-D camera contains RGB information and corresponding depth information. Among them, the imaging range of the above-mentioned RGB-D camera should cover the body parts of the doctor and the patient during the operation, the position area of ​​the upper limb support plate, the fixed area of ​​the operating table and the position area of ​​the C-arm head, and ensure that there is no obstruction to ensure that the above-mentioned surgical scene image that meets the requirements is captured.

[0035] Preferably, after the acquisition module acquires the surgical scene images in real time via the RGB-D camera, it can also perform noise reduction processing on the acquired surgical scene images and transmit the noise-reduced surgical scene images to the processing module 120. The processing module can analyze the status of various intraoperative regions based on the frame sequence of the acquired surgical scene images, thereby dynamically adjusting the position of the patient's left upper limb. It should be noted that the acquisition frequency of the RGB-D camera can be set based on the needs of the actual scenario.

[0036] In the embodiment of the present application, the upper limb support plate area is the position area of ​​the upper limb support plate corresponding to the surgical scene image; the upper limb support plate is an adjustable multi-joint support plate with an upper limb fixing device. For example, the schematic diagram of the upper limb support plate is as shown in FIG. Figure 2 As shown, it is connected to the operating table and is used to place and fix the patient's left upper limb. In addition, the embodiment of the present application can also adjust the position of the upper limb support plate through the multi-axis electric push rod bracket, thereby achieving the effect of adjusting the position of the patient's left upper limb. For example, the upper limb support plate can be fixed to the multi-axis electric push rod bracket, and the surgeon can set the preset adjustable range based on clinical experience and actual needs, such as Figure 2 As shown, the multi-axis electric push rod bracket supports the upper limb support plate to move forward and backward (X axis), left and right (Y axis), up and down (Z axis), and pitch (pitch axis). ), left and right rotation (yaw axis ).

[0037] In addition, the multi-axis electric push rod bracket provided in the acquisition module of the embodiment of the present application not only dynamically adjusts the upper limb support plate based on the target adjustment value obtained by the processing module, but also provides an intraoperative manual adjustment function, so that the surgeon can adjust the position of the above-mentioned upper limb support plate by manually adjusting the multi-axis electric push rod bracket.

[0038] In an embodiment of the present application, the above-mentioned vital sign data are data used to characterize the patient's current physical condition. For example, the vital sign data may include basic health data such as the patient's height and weight, as well as pressure data of the contact position between the patient's left upper limb and the upper limb support plate fixing device during surgery, posture data of the patient's left upper limb, and intraoperative limb status data such as the patient's hand fingertip blood perfusion index. The above-mentioned intraoperative limb status data can be used to monitor the position, angle, pressure distribution, blood flow, etc. of the patient's left upper limb. Taking these data into consideration during the dynamic adjustment process can ensure that the position of the patient's left upper limb is adjusted in time to avoid the patient's discomfort such as numbness, soreness or tenderness in the left upper limb due to maintaining the same posture for a long time.

[0039] Exemplarily, the acquisition module of the embodiment of the present application can obtain the above-mentioned intraoperative limb status data by setting corresponding sensors in the radial artery puncture fixation device. Specifically, it can be implemented as follows: a pressure sensor is set on the surface of the upper limb support plate of the radial artery puncture fixation device to monitor the pressure data of the contact position between the patient's left upper limb and the upper limb support plate during the operation; an inertial measurement unit is deployed on the radial artery puncture fixation device to monitor the posture data of the patient's left upper limb in real time, and the posture data includes three-axis acceleration data and three-axis angular velocity data; a photoplethysmography sensor is installed in the radial artery puncture fixation device to monitor the blood perfusion status of the fingertips of the hand. Among them, the acquisition frequency of each of the above-mentioned sensors is set based on the needs of the actual scenario. Exemplarily, the acquisition frequency of the sensor can be set to 10Hz, and the corresponding data collected by each sensor during the operation is transmitted to the processing module 120.

[0040] At this point, the multimodal adaptive positioning system for the left upper limb of a patient during coronary intervention provided in the embodiment of the present application obtains the image frame sequence of the above-mentioned surgical scene images and the patient's vital sign data during the operation through the above-mentioned acquisition module.

[0041] The above-mentioned processing module 120 is used to determine the initial placement position of the patient's left upper limb on the upper limb support tray based on the vital sign data; and, based on the surgical scene image, determine the surgical operation period and its corresponding surgical operation space area, and determine the degree of interference of the position of the upper limb support tray area on the surgeon's intraoperative operation based on the overlapping pixel points of the upper limb support tray area and the surgical operation space area in each surgical scene image during the surgical operation period and the surgeon's position movement factor. The above-mentioned position movement factor includes a horizontal movement factor and a longitudinal movement factor. The horizontal movement factor is used to characterize the surgeon's lateral position change, and the longitudinal movement factor is used to characterize the surgeon's longitudinal position change; determine the patient's upper limb compression index during the surgical operation period based on the characteristic data; determine the target adjustment value based on the intraoperative operation interference degree and the upper limb compression index.

[0042] The multimodal adaptive positioning system for the left upper limb of a patient during coronary intervention provided in the embodiment of the present application can determine the placement position of the patient's left upper limb through the processing module after the acquisition module acquires the image frame sequence of the surgical scene image and the patient's vital sign data. The specific description is as follows:

[0043] During the preoperative preparation phase of coronary artery intervention, the patient lies supine with the midline of the torso aligned with the long axis of the operating table, the head slightly tilted to the right to expand the left operative field, and the patient's left upper limb is fixed to the upper limb support plate. Before entering the surgery, the processing module determines the initial placement position of the patient's left upper limb on the upper limb support plate. Specifically, the process of determining the initial placement position can be as follows:

[0044] S1: Obtain the initial ideal position of the patient's left upper limb.

[0045] In the embodiments of the present application, the aforementioned initial ideal placement position is the ideal placement position of the patient's left upper limb at the initial moment of surgery, established based on clinical experience. Typically, the initial ideal placement position is set as follows: the patient lies supine, with the left upper limb abducted to an angle of 20° to 35°, palm facing upward, wrist secured to an armrest, wrist slightly dorsiflexed, and elbow naturally extended. This position can effectively improve the success rate of puncture.

[0046] However, due to differences in weight between patients, and the greater average thickness of fat deposits in the supraclavicular fossa and chest wall of obese patients, the anatomical distance between the subclavian artery and the coronary artery opening may be increased. Therefore, during coronary intervention, the left upper limb abduction angle should also be adjusted to a relatively high position. In this case, the upper limb abduction angle needs to be increased to shorten the subclavian artery-coronary artery path. Taking into account the impact of the above situation, after obtaining the above-mentioned initial ideal placement position, the embodiment of the present application further modifies the initial ideal placement position according to the patient's height and weight data through the following step S2 to obtain the patient's final initial placement position of the left upper limb, which is specifically implemented as follows:

[0047] S2: Determine the initial placement of the patient's left upper limb based on height and weight data.

[0048] Exemplarily, the above-mentioned determination of the initial placement position of the patient's left upper limb based on height and weight data can be implemented as follows: obtain the patient's height data and weight data; determine the patient's body mass index (BMI) based on the height data and weight data; obtain the target angle range of the patient's left upper limb abduction, and the target index range consisting of the normal range and overweight range of the body mass index, and determine the corresponding coefficients of the target angle range and the target index range; determine the initial placement angle based on the patient's body mass index and the corresponding coefficient; obtain the support fulcrum of the multi-axis electric push rod bracket for the upper limb support plate area, and the proximal end point of the upper limb support plate area closest to the operating table area, and determine the distance between the support fulcrum and the proximal end point; determine the initial placement position based on the initial placement angle and the distance between the support fulcrum and the proximal end point.

[0049] Specifically, the normal range of the above-mentioned BMI index is usually 18.5~23.9kg / m², and the overweight range is usually 24.0~27.9kg / m². Therefore, the above-mentioned target index range composed of the normal range and overweight range of body mass index is [18.5, 27.9], and the target angle range of the above-mentioned patient's left upper limb abduction is the angle range of [20, 35] of the above-mentioned initial ideal placement position; after determining the above-mentioned target index range and target angle range, for example, the BMI index within the target index range can be mapped to the corresponding angle value within the target angle range based on the interval length, and the corresponding coefficient between the BMI index in the target index range and the left upper limb abduction angle in the target angle range is determined accordingly. Assuming that the maximum abduction range of the patient's left upper limb is 45°, the initial placement angle The specific calculation can be obtained through the following formula:

[0050]

[0051] in, is the initial placement angle of the patient's left upper limb, is the patient's BMI index, is the corresponding coefficient between the BMI index in the target index range and the left upper limb abduction angle in the target angle range; in the above formula, when the calculated initial placement angle When it is less than 20°, take the initial placement angle The value is 20°. When the calculated initial placement angle When it is greater than 45°, take the initial placement angle The value is 45°. When the calculated initial placement angle Between 20° and 45° (inclusive), take the initial placement angle The value of is the calculated value.

[0052] The initial placement angle of the patient is calculated through the above process. After that, it is necessary to further determine the left and right push rod strokes of the multi-axis electric push rod bracket Specifically, the left and right push rod strokes It can be calculated by the following formula: ;in, It is the distance between the support fulcrum of the upper limb support plate area of ​​the above-mentioned multi-axis electric push rod bracket and the proximal end point of the upper limb support plate area and the operating table area.

[0053] Furthermore, the left and right push rod strokes of the multi-axis electric push rod bracket are calculated. After that, the embodiment of the present application can also calculate the left and right push rod strokes The swing module is input to adjust the patient's left upper limb to an initial placement position through the swing module.

[0054] Since most coronary interventions only perform local skin and subcutaneous tissue anesthesia at the puncture point, and other parts of the left upper limb are not anesthetized, the patient is conscious and the limb motor function is fully preserved; in addition, as the operation progresses, the position of the upper limb support plate may hinder the operator's operation path and affect the operator's operation. Therefore, in order to improve the comfort of the patient's left upper limb as much as possible without hindering the doctor's intraoperative operation, the above-mentioned processing module of the embodiment of the present application is not only used to determine the initial placement position of the patient's left upper limb in the surgical preparation stage, but also to dynamically adjust the placement position of the patient's left upper limb during the operation according to the patient's status and the operator's operation. Specifically, the process of dynamically adjusting the placement position of the patient's left upper limb during the operation can be achieved as follows:

[0055] S1: Determine the surgical operation period and its corresponding surgical operation space area based on the surgical scene image.

[0056] In the embodiment of the present application, the above-mentioned surgical operation period refers to the effective surgical operation time period identified from a continuous surgical scene image frame sequence by analyzing the operator's movement characteristics and the spatial relationship of the instrument; the above-mentioned surgical operation spatial area refers to a dynamic set of all effective operation areas within the corresponding surgical operation period, reflecting the spatial range and path of the operator's real-time operation.

[0057] Exemplarily, the above-mentioned determination of the surgical operation time period and its corresponding surgical operation space area based on the surgical scene image can be implemented as follows: obtain an image sequence of surgical scene images collected within the target historical time period; identify the surgeon area corresponding to the surgeon, the patient area corresponding to the patient, the upper limb support plate area and the instrument area in each surgical scene image in the image sequence based on image detection and semantic segmentation algorithms; for each frame of the surgical scene image in the image sequence, determine the basic surgical operation space area based on the position information of the surgeon area and the instrument area in the surgical scene image; for each frame of the surgical scene image in the image sequence, determine the corresponding surgical operation time period based on the surgeon's upper limb skeleton key points in the current surgical scene image frame and the next surgical scene image frame, as well as the surgeon's upper limb skeleton key points and the patient's wrist key points in the current surgical scene image frame; determine the surgical operation space area corresponding to the surgical operation time period based on the surgical operation time period and the basic surgical operation space area.

[0058] The embodiments of the present application can use a deep learning model to perform region recognition and segmentation on the collected surgical scene images, so as to realize the above-mentioned recognition of the operator region corresponding to the surgeon, the patient region corresponding to the patient, the upper limb support plate region, and the instrument region in each surgical scene image in the image sequence based on the image detection and semantic segmentation algorithm. Specifically, the following can be achieved:

[0059] S11: Obtain a historical surgical scene image captured by a top-down RGB-D camera, and mark the surgeon area, patient area, upper limb support plate area, and instrument area in the historical surgical scene image.

[0060] S12: Detection and segmentation of surgeons and patients are achieved through joint pre-training of a cross-platform machine learning framework and a semantic segmentation model.

[0061] In an embodiment of the present application, detection and segmentation of the operator and the patient are achieved through joint pre-training of MediaPipe (a cross-platform machine learning framework) and YOLOv8-seg (YOLOv8 semantic segmentation model); among them, MediaPipe is used for real-time skeleton point detection of the operator and the patient to obtain the coordinates of skeleton points such as the shoulder, wrist, and elbow, limit the segmentation of the region of interest (ROI), and reduce the computational burden of the segmentation model.

[0062] Exemplarily, the above-mentioned joint pre-training can be implemented as follows: the above-mentioned historical surgical scene images collected are divided into training set, validation set, and test set in the ratio of 80%, 10%, and 10% respectively; the detection segmentation targets are set to the surgeon as a whole and the patient as a whole, and data enhancement processing is performed, and the above-mentioned data enhancement may include random rotation, scaling, reflection and noise processing; the AdamW optimizer is set, and the complete intersection over Union Loss (CIoU Loss) and the binary cross entropy loss function (BCELoss) are set as the loss functions in the training process; the model pre-weight is loaded through transfer learning; after training, verification, and testing, the weight model with the highest mean average precision (mAP) of the category detection results and the intersection over Union (IoU) of the predicted box and the real box in the target detection is selected to complete the training.

[0063] S13: A deep semantic segmentation network is used to achieve detailed analysis of the upper limb support plate area and surgical operation area.

[0064] Exemplarily, the detailed analysis of the upper limb support plate area and surgical operation area using a deep semantic segmentation network can be achieved as follows: Surgical scene images are captured during the procedure, the upper limb support plate area and the instrument area are annotated, and data augmentation processing is performed; an Adam optimizer is configured, and the Dice Loss function and the Cross-Entropy Loss function are set as loss functions; after training, verification, and testing, the model accuracy is evaluated based on a preset threshold of the Dice coefficient to complete the training. Exemplarily, the preset threshold of the Dice coefficient can be 0.8.

[0065] S14: Identify the operator area, patient area, upper limb support plate area, and instrument area in the surgical scene image.

[0066] In an embodiment of the present application, this step is used to integrate and deploy the model obtained by the above training to realize the above-mentioned area recognition function. Specifically, it can be achieved as follows: input the surgical scene image, and use MediaPipe+YOLOv8 to run in real time to obtain the contours of the surgeon and patient, and segment and crop the region of interest; input the region of interest and the corresponding surgical scene image into DeepLabv3, and segment the upper limb support plate area and the surgical operation area; output the masks of the upper limb support plate area and the surgical operation area and superimpose them on the original surgical scene image to realize intraoperative image perception, and obtain the surgeon area, patient area, upper limb support plate area and instrument area in the surgical scene image.

[0067] After identifying the operator area, patient area, upper limb support plate area and instrument area in each surgical scene image within a certain time period through the above steps S11 to S14, illustratively, for each frame of the surgical scene image in the image sequence, determining the basic area of ​​the surgical operation space based on the position information of the operator area and the instrument area in the surgical scene image can be achieved as follows: for each frame of the surgical scene image in the image sequence, determining the instrument area adjacent to the operator area in the current surgical scene image frame, calculating the average value of the depth value difference between the adjacent pixel points of the operator area and the adjacent instrument area; determining the current operating instrument area based on the average value of the depth value difference between the adjacent pixel points, merging the operator area and the current operating instrument area to obtain the basic area of ​​the surgical operation space. The above basic area of ​​the surgical operation space refers to the static spatial combination area of ​​the operator and the current operating instrument during the operation, which is used to evaluate the direct interference of the upper limb support plate area on the operator's operation.

[0068] In a specific embodiment, the process of determining the basic area of ​​the surgical operation space is described in detail below:

[0069] Specifically, the above-mentioned determination of the basic area of ​​the surgical operation space can be achieved as follows: obtain a sequence of surgical scene image frames collected within the past 10 seconds, and the results of region recognition and segmentation of each frame of the surgical scene image; for each frame of the surgical scene image, determine all the instrument areas adjacent to the surgeon area in the current surgical scene image frame, and calculate the average value of the depth difference between adjacent pixels between the surgeon area and the adjacent instrument areas. For example, assuming that there are two instrument areas adjacent to the surgeon area 1 in the current surgical scene image frame, namely instrument area 1 and instrument area 2, then calculate the depth value difference between adjacent pixels of the surgeon area 1 and instrument area 1, and the depth value difference between adjacent pixels of the surgeon area 1 and instrument area 2. If there are 5 pairs of adjacent pixels between the surgeon area 1 and instrument area 1, and 3 pairs of adjacent pixels between the surgeon area 1 and instrument area 2, then the average value of the depth difference between adjacent pixels between the surgeon area and the adjacent instrument area is the average of the depth value differences between these 8 pairs of adjacent pixels, that is, ; Set the judgment threshold of the operating device ,Will Less than The adjacent instrument area is used as the current instrument area; the operator area and the current instrument area are combined to form the basic area of ​​the surgical operation space. The current instrument area is the area where the surgical instruments used by the operator during the operation are located and is also part of the surgical operation space.

[0070] In the above process of determining the basic area of ​​the surgical operation space, in order to ensure that the identified surgical instruments are indeed used by the surgeon, the processing module of the embodiment of the present application combines the spatial connectivity characteristics and depth data of the surgical scene image, dynamically evaluates the spatial relationship between the surgeon area and the instrument area, and screens out the instrument area that is directly connected to the surgeon area and is within a reasonable operating depth range, avoiding the misjudgment of other non-operational instruments or equipment entering the basic area of ​​the surgical operation space.

[0071] After determining the basic area of ​​the surgical operation space, the embodiment of the present application can further determine the above-mentioned surgical operation time period and its corresponding surgical operation space area, which can be specifically implemented as follows:

[0072] During actual coronary intervention, in addition to surgical procedures such as guidewires and catheters, the surgeon also performs non-surgical procedures such as body position adjustment, instrument preparation, C-arm adjustment, and coordination with assistants. When determining the surgical operation period and its corresponding surgical operation space, the interference of these non-surgical operations should be eliminated as much as possible. Because the surgeon's main operating area during coronary intervention is highly concentrated around the radial artery puncture point, when dynamically evaluating the surgical operation space, the surgeon's movement trends can be combined with a focus on analyzing the characteristics of their movements toward the radial artery puncture point to accurately determine the surgeon's actual operation path and spatial range, avoiding interference from these non-surgical operations.

[0073] Exemplarily, determining the surgical operation time period can be achieved as follows: obtaining a first shoulder line between a shoulder key point of the surgeon in a current surgical scene image frame and a shoulder key point of the surgeon in a next surgical scene image frame, as well as a second shoulder line between a shoulder key point of the surgeon in the current surgical scene image frame and a wrist key point of the patient, and determining a first angle between the first shoulder line and the second shoulder line; obtaining a first wrist line between a wrist key point of the surgeon in the current surgical scene image frame and a wrist key point of the surgeon in the next surgical scene image frame, as well as a second wrist line between a wrist key point of the surgeon in the current surgical scene image frame and a wrist key point of the patient, and determining a first angle between the first shoulder line and the second shoulder line. The method comprises the steps of: obtaining a first elbow line between the surgeon's elbow key point in the current surgical scene image frame and the surgeon's elbow key point in the next surgical scene image frame, and obtaining a second elbow line between the surgeon's elbow key point and the patient's wrist key point in the current surgical scene image frame, and determining a third angle between the first elbow line and the second elbow line; calculating an average value between the first angle, the second angle and the third angle as a surgical operation path evaluation coefficient; sorting the surgical operation path evaluation coefficients of each frame of the surgical scene image in the image sequence according to size, and determining the surgical operation time period based on the obtained surgical operation path evaluation coefficient sequence.

[0074] In an embodiment of the present application, the above-mentioned surgical operation path evaluation coefficient is used to measure the characteristics of the surgeon's surgical operation path during the operation, and reflects the surgeon's movement trend during the operation by determining the surgeon's joint movement direction between consecutive surgical scene image frames and the relative movement direction between the surgeon's joints and the patient's wrist; wherein, the above-mentioned surgical operation path refers to the effective space for the surgeon's hands and instruments to move when operating guide wires, catheters, etc.

[0075] After determining the above-mentioned surgical operation path evaluation coefficient, the surgical operation time period can be determined based on the obtained surgical operation path evaluation coefficient. Specifically, the process of determining the surgical operation time period can be implemented as follows: sort the surgical operation path evaluation coefficients corresponding to each surgical scene image frame in order from large to small to obtain a surgical operation path evaluation coefficient sequence; calculate the difference between adjacent elements in the surgical operation path evaluation coefficient sequence, obtain the larger element among the adjacent elements with the largest difference and use its value as the candidate threshold; determine the surgical scene image frames in each surgical scene image frame whose surgical operation path evaluation coefficient is greater than or equal to the candidate threshold, and use the time range of the selected surgical scene image frames as the surgical operation time period.

[0076] In a specific embodiment, taking the first frame of surgical scene image to the eighth frame of surgical scene image as an example, assuming that the acquired surgical operation path evaluation coefficients are 38, 55, 23, 35, 45, 30, 57, and 32 respectively, the surgical operation path evaluation coefficients corresponding to the above surgical scene image frames are sorted, and the obtained surgical operation path evaluation coefficient sequence is [23, 30, 32, 35, 38, 45, 55, 57]; the difference between adjacent elements in the surgical operation path evaluation coefficient sequence is calculated, and the obtained difference sequence is [7, 2, 3, 3, 7, 10, 2], among which the adjacent element pair with the largest difference is [45, 55], and the larger element in the adjacent element pair is 55. The above surgical operation path evaluation coefficients that are greater than or equal to the candidate thresholds include 55 and 57, which correspond to the second frame of surgical scene image and the seventh frame of surgical scene image, respectively, and the time from the second frame of surgical scene image to the seventh frame of surgical scene image is taken as the surgical operation period.

[0077] In an embodiment of the present application, after determining the above-mentioned surgical operation time period, further, by analyzing the position change characteristics of the surgeon and other instrument areas over a period of time in the past, the surgical operation space area corresponding to the current surgical operation time period is determined. Exemplarily, the above-mentioned determination of the surgical operation space area corresponding to the surgical operation time period can be achieved as follows: for each surgical operation time period, the frame difference pixel points of the surgical operation space basic area in the current surgical scene image frame and the next surgical scene image frame in the surgical operation time period are obtained, and the convex hull formed based on the frame difference pixel points is used as the surgical operation space area corresponding to the surgical operation time period.

[0078] The frame difference pixels are pixels in the basic region of the surgical operation space that change between adjacent surgical scene image frames and are used to characterize positional changes in the operator and other instrument regions. For example, pixels that belong to the basic region of the surgical operation space in the current surgical scene image frame but do not belong to the basic region of the surgical operation space in the next surgical scene image frame, or pixels that do not belong to the basic region of the surgical operation space in the current surgical scene image frame but do belong to the basic region of the surgical operation space in the next surgical scene image frame.

[0079] Specifically, assuming that the current surgical operation period corresponds to 10 consecutive surgical scene image frames, the above-mentioned determination of the surgical operation space area corresponding to the current surgical operation period can be achieved as follows: for each of the 10 consecutive surgical scene image frames, the frame difference pixel points of the surgical operation space basic area are calculated; the convex hull formed by all the frame difference pixel points of the adjacent surgical scene image frames in the 10 consecutive surgical scene image frames is used as the surgical operation space area of ​​the current surgical operation period.

[0080] S2: Based on the overlapping pixel points of the upper limb support plate area and the surgical operation space area in each surgical scene diagram during the surgical operation period and the surgeon's position movement factor, determine the degree of interference of the position of the upper limb support plate area on the surgeon's intraoperative operation. The above-mentioned position movement factor includes a horizontal movement factor and a longitudinal movement factor. The horizontal movement factor is used to characterize the surgeon's lateral position change, and the longitudinal movement factor is used to characterize the surgeon's longitudinal position change.

[0081] In an embodiment of the present application, after determining the surgical operation time period and its corresponding surgical operation space area, the degree of interference of the upper limb support plate area with the surgeon's intraoperative operation can be evaluated based on the degree of overlap between the upper limb support plate area and the surgical operation space area. The greater the degree of overlap, the greater the degree of interference of the position of the upper limb support plate with the surgeon's operation.

[0082] Exemplarily, determining the degree of overlap between the upper limb support plate area and the surgical operation space area can be achieved by the following method: for each frame of the surgical scene image within the surgical operation period, obtaining the overlapping pixel points of the upper limb support plate area and the surgical operation space area in the surgical scene image, and recording the position corresponding to the overlapping pixel points as the potential interference position; in the surgical scene image frames other than the current surgical scene image frame within the surgical operation period, obtaining the target position where the pixel points at the potential interference position are no longer the overlapping pixel points, calculating the mean depth value of the pixel points at the target position of the other surgical scene image frames, and calculating the difference between the mean depth value and the depth value of the pixel points at the potential interference position to obtain the interference error corresponding to the potential interference position; calculating the average value of the interference error corresponding to each potential interference position to obtain the depth difference factor corresponding to the current surgical scene image frame, and the depth difference factor is used to measure the degree of overlap between the above-mentioned upper limb support plate area and the surgical operation space area.

[0083] Specifically, taking the current surgical operation period including 5 frames of continuous surgical scene images as an example, the first frame of the surgical scene image is taken as the current surgical scene image frame. In the other 4 frames of surgical scene images from the 2nd frame to the 5th frame, if there is no upper limb support plate pixel point at the corresponding position of a potential interference position in the 1st frame of the surgical scene image, the corresponding position of the potential interference position in the surgical scene image frame is recorded as the above-mentioned target position; the depth values ​​of the pixel points at the target position corresponding to the potential interference position in the 4 frames of surgical scene images from the 2nd frame to the 5th frame are extracted and averaged; the difference between the average value of the above depth values ​​and the depth value of the pixel points at the potential interference position in the 1st frame of the surgical scene image is calculated to obtain the interference error corresponding to the potential interference position; the average value of the interference error of each potential interference position is calculated to obtain the depth difference factor corresponding to the current surgical scene image frame.

[0084] In addition, during the actual intraoperative operation, if the surgeon's movements are restricted or the operation is laborious, it may cause body shaking or abnormal posture during the operation, thereby interfering with the smooth progress of the operation. Therefore, the multimodal coronary intervention-based adaptive positioning system for the patient's left upper limb during dynamic adjustment of the position of the upper limb support plate provided in the embodiment of the present application, in addition to evaluating the direct interference of the upper limb support plate on the surgical operation space area through the above process, it is also necessary to analyze the surgeon's operating load during the operation, dynamically optimize the position of the upper limb support plate, and improve the convenience and comfort of the surgeon's operation.

[0085] Among them, the stability of the above-mentioned surgeon's movements and the operation load can be quantified by analyzing the displacement characteristics of the surgeon's body area in the frame-by-frame surgical scene images during the surgical operation period. Exemplarily, the quantification process can be implemented as follows: for each frame of the surgical scene image during the surgical operation period, based on the position information and depth information of the surgeon's upper limb skeleton key points in the current surgical scene image frame and the next surgical scene image frame, determine the surgeon's horizontal movement factor and longitudinal movement factor corresponding to the current surgical scene image frame.

[0086] Specifically, the process of determining the above-mentioned operator's horizontal movement factor can be implemented as follows: obtain the operator's skeleton key points (including shoulder key points, wrist key points and elbow key points) in each frame of the surgical scene image within the current surgical operation period; for each frame of the surgical scene image, determine the vectors pointing from the shoulder key points, wrist key points and elbow key points in the current surgical scene image frame to the shoulder key points, wrist key points and elbow key points in the next surgical scene image frame, and calculate their average vector as the operator's horizontal displacement vector corresponding to the current surgical scene image frame; calculate the vector cosine similarity of the horizontal displacement vectors corresponding to the current surgical scene image frame and the next surgical scene image frame, and record it as the horizontal displacement vector cosine similarity of the current surgical scene image frame; calculate the average value of the horizontal displacement vector cosine similarities of all surgical scene images in the current surgical operation period; calculate the absolute value of the difference between the horizontal displacement vector cosine similarity of the current surgical scene image frame and the average value of the horizontal displacement vector cosine similarities of all surgical scene images in the current surgical operation period, and record it as the operator's horizontal movement factor corresponding to the current surgical scene image frame.

[0087] The above process of determining the operator's longitudinal movement factor can be implemented as follows: obtain the operator's various skeleton key points (including shoulder key points, wrist key points and elbow key points) in each frame of the surgical scene image during the current surgical operation period; for each frame of the surgical scene image, calculate the difference between the depth value of the operator's skeleton key point in the current surgical scene image frame and the depth value of the corresponding operator's skeleton key point in the next surgical scene image frame, and record it as the operator's longitudinal displacement corresponding to the current surgical scene image frame. For example, since the wrist directly corresponds to the execution point of the operator's operation, its depth change can better reflect the longitudinal displacement of the actual surgical operation. Therefore, in the above longitudinal displacement, the wrist key point can be preferably selected; for each frame of the surgical scene image during the current surgical operation period, , calculate the rate of change of the surgeon's longitudinal displacement from the current surgical scene image frame to the next adjacent surgical scene image frame (that is, the ratio of the difference between the surgeon's longitudinal displacement of the current surgical scene image frame and the next adjacent surgical scene image frame to the time interval between these two adjacent surgical scene images), and record it as the surgeon's longitudinal displacement change rate of the current surgical scene image frame; calculate the average value of the surgeon's longitudinal displacement change rate of each surgical scene image frame within the current surgical operation period; calculate the absolute value of the difference between the surgeon's longitudinal displacement change rate of the current surgical scene image frame and the average value of the surgeon's longitudinal displacement change rate of each surgical scene image frame within the current surgical operation period, and record it as the surgeon's longitudinal movement factor corresponding to the current surgical scene image frame.

[0088] In the embodiment of the present application, after determining the depth difference factor of each surgical scene image and the surgeon's horizontal movement factor and vertical movement factor during the current surgical operation period through the above process, specifically, the interference degree of the upper limb support plate position on the surgeon's intraoperative operation during the current surgical operation period can be calculated by the following formula:

[0089]

[0090] in, The degree of interference of the upper limb support plate position on the surgeon's intraoperative operation during the current surgical operation period; The number of image frames of the surgical scene image collected during the current surgical operation period; is the surgeon's horizontal movement factor corresponding to the i-th frame of the surgical scene image, which is used to evaluate the degree of limb shaking during the surgeon's operation from a horizontal perspective; is the longitudinal movement factor of the surgeon corresponding to the i-th frame of the surgical scene image, which is used to evaluate the degree of limb shaking during the surgeon's operation from a longitudinal perspective; and The larger the value, the more the operator's limbs have shaking fluctuation characteristics during the current surgical operation period, and the greater the operating load caused by the current upper limb support plate position on the operator; is the depth difference factor corresponding to the i-th frame of the surgical scene image; The function is used for normalization.

[0091] In an embodiment of the present application, after determining the degree of interference of the position of the upper limb support plate with the operator's intraoperative operation during the current surgical operation period through the above process, it is illustratively possible to determine whether the position of the upper limb support plate needs to be adjusted by setting an adjustment threshold. For example, assuming that the above adjustment threshold is 0.6, when the calculated value of the degree of interference of the position of the upper limb support plate with the operator's intraoperative operation during the current surgical operation period is greater than 0.6, it is determined that the position of the upper limb support plate needs to be adjusted.

[0092] S3: Determine the patient's upper limb compression index during the surgical operation period based on the characteristic data.

[0093] In an embodiment of the present application, if the intraoperative operation interference level at the current moment determined by the above process is usually less than the adjustment threshold, but the target time has passed since the last adjustment of the position of the upper limb support plate (for example, the target time can be set to 10 minutes), then in order to reduce the patient's discomfort, it is necessary to further determine whether the position of the upper limb support plate needs to be adjusted in combination with the patient's vital signs data to prevent the patient's upper limb from being in the same position for too long, compressing the arteries and veins, thereby leading to insufficient distal blood perfusion and causing discomfort in the patient's upper limbs.

[0094] Exemplarily, the above-mentioned determination of the patient's upper limb compression index during the surgical operation period based on characteristic data can be achieved as follows: the pressure data of the patient at each time point during the surgical operation period is obtained through each pressure sensor on the upper limb support plate area and the average is calculated; the blood perfusion index of the patient at each time point during the surgical operation period is obtained through the photoelectric volumetric pulse wave sensor on the upper limb support plate area; at each time point during the surgical operation period, the ratio of the pressure mean and the blood perfusion index is calculated to obtain the upper limb compression index.

[0095] In a specific embodiment, the process of determining the upper limb compression index can be implemented as follows: obtaining the monitoring data of each pressure sensor and photoelectric volumetric pulse wave sensor in the upper limb support plate during the static time period from the time when the upper limb support plate was last adjusted to the current time; obtaining the blood perfusion index (Perfusion Index, PI) at each time point in the above-mentioned static time period based on the monitoring data of the photoelectric volumetric pulse wave sensor; obtaining the average pressure value corresponding to each time point in the above-mentioned static time period for each pressure sensor; at each time point in the above-mentioned static time period, dividing the pressure average value by the blood perfusion index, and setting the value range to [0,1] through the tanh function, to obtain the upper limb compression index at each time point in the above-mentioned static time period. .

[0096] In the embodiment of the present application, if the upper limb compression index If the preset conditions are met, even if the intraoperative manipulation degree does not reach the adjustment threshold, the position of the upper limb support plate needs to be adjusted. For example, after the position of the upper limb support plate is adjusted due to upper limb discomfort, the upper limb compression index shows a decreasing trend. As the upper limb support plate is fixed for an increasing period of time, the upper limb compression index stops decreasing and further increases over time. As the blood perfusion index decreases to a relatively stable value, the growth of the upper limb compression index gradually slows down. At this time, the position of the upper limb support plate needs to be adjusted.

[0097] Specifically, the above-mentioned preset conditions can be determined as follows: with the upper limb compression index as the vertical axis and time as the horizontal axis, the upper limb compression index at each time point in the above-mentioned static time period is fitted to obtain the corresponding upper limb compression index fitting curve, and the derivative corresponding to each upper limb compression index-time data point on the upper limb compression index fitting curve is calculated; if the derivative corresponding to the upper limb compression index at the current time point is less than the derivative mean of the monitoring data of the above-mentioned static time period, and the upper limb compression index is greater than or equal to the maximum value of the upper limb compression index in the longest increase interval of the monitoring data of the above-mentioned static time period, it is determined that the upper limb support plate needs to be adjusted at the current moment.

[0098] S4: Determine the target adjustment value based on the intraoperative operation interference level and upper limb compression index.

[0099] In an embodiment of the present application, when adjusting the position of the upper limb support plate, the adjustment direction of the upper limb support plate should be away from the surgical operation space area while ensuring the distance between the radial artery puncture point and the operator as much as possible.

[0100] Exemplarily, when the intraoperative operation interference degree is greater than the adjustment threshold or the upper limb compression index meets the above-mentioned preset conditions, the above-mentioned determination of the target adjustment value based on the intraoperative operation interference degree and the upper limb compression index can be implemented as follows: based on the adjustable direction of the multi-axis electric push rod bracket and the distance between the patient's wrist key point and the surgeon's shoulder key point, the target adjustment direction of the upper limb support plate area is determined; the intraoperative operation interference degree or the upper limb compression index is used as the adjustment coefficient, and the target adjustment value is determined based on the adjustment coefficient and the preset adjustable range of the upper limb support plate area.

[0101] In the embodiment of the present application, the adjustable direction of the multi-axis electric push rod bracket is any adjustable direction that the multi-axis electric push rod bracket can achieve.

[0102] In a specific embodiment, the above-mentioned determination of the target adjustment value based on the intraoperative operation interference degree and the upper limb compression index can be implemented as follows: obtaining the adjustable direction set of the multi-axis electric push rod bracket ; Exclude adjustable direction set The direction that will cause the upper limb support plate to approach the surgical operation space area is obtained, and the candidate direction set is obtained. ; For the candidate direction set For each candidate direction in the , calculate the distance change value between the patient's wrist key point and the operator's shoulder key point corresponding to each moving unit pixel, and determine the candidate direction with the smallest distance change value as the target adjustment direction of the upper limb support plate; if the reason for this adjustment is that the intraoperative operation interference is greater than the adjustment threshold, the above intraoperative operation interference is As the adjustment coefficient A, if the reason for this adjustment is that the upper limb compression index meets the preset conditions, the upper limb compression index As the adjustment coefficient A, the target adjustment value is calculated by the following formula: ,in, It is the difference between the maximum value of the preset adjustable range of the upper limb support plate in the target adjustment direction and the current position of the upper limb support plate. The above preset adjustable range is set by the operator based on the actual scenario.

[0103] At this point, the embodiment of the present application determines the initial placement position of the patient's left upper limb in the preparation stage of coronary artery interventional treatment, as well as the dynamic adjustment direction and adjustment value of the upper limb support plate during the operation through the above process.

[0104] The positioning module 130 is used to input the target adjustment value into the multi-axis electric push rod support controller, so as to control the multi-axis electric push rod support to adjust the upper limb support plate area to the target position through the multi-axis electric push rod support controller.

[0105] Exemplarily, the above-mentioned control of the multi-axis electric push rod bracket by the multi-axis electric push rod bracket controller to adjust the upper limb support plate area to the target position can be achieved as follows: the multi-axis electric push rod bracket controller controls the multi-axis electric push rod bracket along the target adjustment direction to move the upper limb support plate area to the target position corresponding to the target adjustment value.

[0106] Specifically, after obtaining the target adjustment value for the upper limb support plate area, the target adjustment value can be input into the multi-axis electric push rod support controller. The multi-axis electric push rod support controller controls the multi-axis electric push rod support to rotate along the target adjustment direction to adjust the upper limb support plate position to the target position indicated by the target adjustment value, thereby achieving adaptive adjustment of the upper limb support plate position. During the above adjustment process, the adjustment speed must also be determined. For example, it is stipulated that the speed during the adjustment process does not exceed 5 cm / s.

[0107] At this point, this embodiment is completed.

[0108] The present invention collects surgical scene images and patient vital sign data through the above-mentioned acquisition module, so that the processing module can determine the initial placement position of the patient's left upper limb based on the collected patient vital sign data, and then adjust the upper limb support plate area to the initial placement position through the multi-axis electric push rod bracket. The patient's characteristic data is taken into account in the process of determining the initial placement position, and a suitable initial placement position can be determined for different patients, which helps to alleviate the patient's intraoperative discomfort; in addition, during the operation, the present invention determines the position of the upper limb support plate area in each surgical operation period based on the collected surgical scene images and the patient's vital sign data through the above-mentioned processing module. The target adjustment value of the upper limb support plate area is determined based on the intraoperative operation interference degree and the upper limb compression index, and the position of the above-mentioned upper limb support plate area is dynamically adjusted based on the determined target adjustment value through the multi-axis electric push rod bracket, so that the discomfort of the patient's surgical side limb during coronary intervention can be reduced on the premise of facilitating the doctor's intraoperative operation, thereby improving the operational efficiency of coronary intervention.

[0109] It should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A multimodal adaptive positioning system for the left upper limb of a patient during coronary intervention, characterized by: The system comprises: An acquisition module is used to acquire surgical scene images and patient vital sign data; the surgical scene images cover the surgeon, patient, operating table area, and upper limb support plate area; A processing module is used to determine the initial placement position of the patient's left upper limb on the upper limb support plate based on the vital sign data; and determine the surgical operation period and its corresponding surgical operation space area based on the surgical scene image; determine the degree of interference of the position of the upper limb support plate area on the surgeon's intraoperative operation based on the overlapping pixel points of the upper limb support plate area and the surgical operation space area in each of the surgical scene images within the surgical operation period and the surgeon's position movement factor, the position movement factor including a horizontal movement factor and a longitudinal movement factor, the horizontal movement factor being used to characterize the lateral position change of the surgeon, and the longitudinal movement factor being used to characterize the longitudinal position change of the surgeon; determine the upper limb compression index of the patient within the surgical operation period based on the feature data; determine the target adjustment value based on the intraoperative operation interference degree and the upper limb compression index; The positioning module is used to input the target adjustment value into the multi-axis electric push rod support controller, so as to control the multi-axis electric push rod support to adjust the upper limb support plate area to the target position through the multi-axis electric push rod support controller.

2. The multimodal adaptive positioning system for the left upper limb of a patient during coronary intervention according to claim 1, characterized in that: The physical sign data includes the patient's height data and weight data; the processing module determines the initial placement position of the patient's left upper limb on the upper limb support plate based on the physical sign data by the following method, including: determining a body mass index of the patient based on the height data and the weight data; Obtaining a target angle range for left upper limb abduction of the patient and a target index range consisting of a normal range and an overweight range of the body mass index, and determining corresponding coefficients for the target angle range and the target index range; Determining the initial placement angle based on the body mass index of the patient and the corresponding coefficient; Obtaining a support point of the multi-axis electric push rod support for the upper limb support plate area and a proximal end point of the upper limb support plate area closest to the operating table area, and determining a distance between the support point and the proximal end point; The initial placement position is determined based on the initial placement angle and the distance between the support fulcrum and the proximal endpoint.

3. The multimodal adaptive positioning system for the left upper limb of a patient during coronary intervention according to claim 1, characterized in that: The processing module implements the determination of the surgical operation time period and its corresponding surgical operation space area based on the surgical scene image by the following method, including: Acquiring an image sequence of the surgical scene images collected within a target historical time period; Identify, based on image detection and semantic segmentation algorithms, the operator area corresponding to the operator, the patient area corresponding to the patient, the upper limb support plate area, and the instrument area in each of the surgical scene images in the image sequence; For each frame of the surgical scene image in the image sequence, determining a basic area of ​​the surgical operation space based on position information of the operator area and the instrument area in the surgical scene image; For each frame of the surgical scene image in the image sequence, determining the corresponding surgical operation time period based on the surgeon's upper limb skeleton key points in the current surgical scene image frame and the next surgical scene image frame, and the surgeon's upper limb skeleton key points and the patient's wrist key points in the current surgical scene image frame; The surgical operation space area corresponding to the surgical operation time period is determined based on the surgical operation time period and the surgical operation space basic area.

4. The multimodal adaptive positioning system for the left upper limb of a patient during coronary intervention according to claim 3, characterized in that: The processing module determines the basic area of ​​the surgical operation space based on the position information of the operator area and the instrument area in each frame of the surgical scene image in the image sequence by the following method, including: For each frame of the surgical scene image in the image sequence, determining the instrument region adjacent to the operator region in the current surgical scene image frame, and calculating an average value of the depth value differences between adjacent pixel points in the operator region and the adjacent instrument region; The current operating instrument area is determined based on the average value of the difference between the depth values ​​of the adjacent pixels, and the operator area and the current operating instrument area are merged to obtain the basic area of ​​the surgical operation space.

5. The multimodal adaptive positioning system for the left upper limb of a patient during coronary intervention according to claim 3, characterized in that: The processing module determines the corresponding surgical operation time period for each frame of the surgical scene image in the image sequence based on the key points of the surgeon's upper limb skeleton in the current surgical scene image frame and the next surgical scene image frame, and the key points of the surgeon's upper limb skeleton and the key points of the patient's wrist in the current surgical scene image frame, by the following method, including: Obtaining a first shoulder line between a shoulder key point of the operator in the current surgical scene image frame and a shoulder key point of the operator in the next surgical scene image frame, and a second shoulder line between a shoulder key point of the operator in the current surgical scene image frame and a wrist key point of the patient, and determining a first angle between the first shoulder line and the second shoulder line; Obtaining a first wrist line between a key point of the surgeon's wrist in the current surgical scene image frame and a key point of the surgeon's wrist in the next surgical scene image frame, and a second wrist line between a key point of the surgeon's wrist in the current surgical scene image frame and a key point of the patient's wrist, and determining a second angle between the first wrist line and the second wrist line; Obtaining a first elbow line between a key point of the surgeon's elbow in the current surgical scene image frame and a key point of the surgeon's elbow in the next surgical scene image frame, and a second elbow line between a key point of the surgeon's elbow in the current surgical scene image frame and a key point of the patient's wrist, and determining a third angle between the first elbow line and the second elbow line; calculating an average value of the first angle, the second angle, and the third angle as a surgical operation path evaluation coefficient; The surgical operation path evaluation coefficients of each frame of the surgical scene image in the image sequence are sorted in order of size, and the surgical operation time period is determined based on the obtained surgical operation path evaluation coefficient sequence.

6. The multimodal adaptive positioning system for the left upper limb of a patient during coronary intervention according to claim 3, characterized in that: The processing module determines the surgical operation space area corresponding to the surgical operation period based on the surgical operation period and the surgical operation space basic area by the following method, including: For each surgical operation period, the frame difference pixel points of the surgical operation space basic area in the current surgical scene image frame and the next surgical scene image frame within the surgical operation period are obtained, and the convex hull formed based on the frame difference pixel points is used as the surgical operation space area corresponding to the surgical operation period.

7. The multimodal adaptive positioning system for the left upper limb of a patient during coronary intervention according to claim 1, characterized in that: The processing module determines the interference degree of the position of the upper limb support plate area on the intraoperative operation of the operator based on the overlapping pixels of the upper limb support plate area and the surgical operation space area in each surgical scene image within the surgical operation period and the position movement factor of the operator by the following method, including: For each frame of the surgical scene image within the surgical operation period, obtaining overlapping pixel points of the upper limb support plate area and the surgical operation space area in the surgical scene image, and recording the positions corresponding to the overlapping pixel points as potential interference positions; In other surgical scene image frames other than the current surgical scene image frame during the surgical operation period, obtaining a target position where the pixel point at the potential interference position is no longer the overlapping pixel point, calculating a mean depth value of the pixel points at the target position in the other surgical scene image frames, and calculating a difference between the mean depth value and the depth value of the pixel point at the potential interference position to obtain an interference error corresponding to the potential interference position; Calculating an average value of the interference error corresponding to each of the potential interference positions to obtain a depth difference factor corresponding to the current surgical scene image frame; For each frame of the surgical scene image within the surgical operation period, based on the position information and depth information of the surgeon's upper limb skeleton key points in the current surgical scene image frame and the next surgical scene image frame, determine the surgeon's horizontal movement factor and the longitudinal movement factor corresponding to the current surgical scene image frame; The intraoperative operation interference degree corresponding to the surgical operation period is determined based on the horizontal movement factor, the longitudinal movement factor, and the depth difference factor.

8. The multimodal adaptive positioning system for the left upper limb of a patient during coronary intervention according to claim 1, characterized in that: The processing module determines the upper limb compression index of the patient during the surgical operation period based on the characteristic data by the following method, including: Obtaining the pressure data of the patient at each time point during the surgical operation period through the pressure sensors on the upper limb support plate area and calculating the average; Obtaining the patient's blood perfusion index at various time points during the surgical operation period through a photoplethysmography sensor on the upper limb support plate area; At each time point during the surgical operation, the ratio of the mean pressure value to the blood perfusion index is calculated to obtain the upper limb compression index.

9. The multimodal adaptive positioning system for the left upper limb of a patient during coronary intervention according to claim 1, characterized in that: The processing module determines the target adjustment value based on the intraoperative operation interference degree and the upper limb compression index by the following method, including: When the average value of the intraoperative operation interference degree in each surgical operation period is greater than a preset threshold, or the upper limb compression index meets a preset condition, a target adjustment direction of the upper limb support plate area is determined based on the adjustable direction of the multi-axis electric push rod bracket and the distance between the patient's wrist key point and the operator's shoulder key point; The intraoperative operation interference degree or the upper limb compression index is used as an adjustment coefficient, and the target adjustment value is determined based on the adjustment coefficient and a preset adjustable range of the upper limb support plate area.

10. The multimodal adaptive positioning system for the left upper limb of a patient during coronary intervention according to claim 1, characterized in that: The positioning module realizes the adjustment of the upper limb support plate area to the target position by controlling the multi-axis electric push rod support controller through the multi-axis electric push rod support controller, including: The multi-axis electric push rod bracket controller controls the multi-axis electric push rod bracket along the target adjustment direction to move the upper limb support plate area to the target position corresponding to the target adjustment value.

Images