Patient sign data verification and restoration method, device and equipment

Abstract

The invention discloses a method, a device and equipment for verifying and repairing physical sign data of a patient. The method comprises the following steps: acquiring the physical sign data of the patient collected by a terminal; acquiring a frame image from the streaming media server, wherein the frame image is a video frame obtained by performing frame analysis on a video stream acquired from the terminal; sampling the content of the frame image to obtain image sampling data; comparing the frequency information of the image sampling data and the sign data of the patient at the same time; and when it is determined that data loss exists in the sign data of the patient, repairing the sign data of the patient by using the frequency information of the image sampling data. Therefore, the problem that the historical physical sign data of the patient is distorted and the real-time performance cannot be ensured due to the packet loss phenomenon in the transmission process of the existing physical sign data of the patient is solved.

Description

Technical Field

[0001] This application relates to the field of remote consultation data transmission technology, and in particular to a method, device and equipment for verifying and repairing patient vital signs data. Background Technology

[0002] With the promotion of domestic wireless communication technology and digital healthcare, remote diagnosis and treatment solutions are constantly improving. Emergency patients can complete necessary physical examinations en route to the hospital and send the data to the hospital's server via the internet, making pre-hospital emergency care possible and significantly reducing the risk of delayed treatment. Local hospitals can also send collected patient vital sign data to higher-level hospitals for remote consultations via the internet. This allows patients to receive treatment services from top-tier hospitals in their local area, reducing not only medical costs but also the risk of treatment complications due to the lack of experience among local doctors.

[0003] As we all know, in the era of big data, the historical vital signs data generated during remote consultations are of great research value to both hospitals and research institutions. They are also an indispensable data element for future model training. However, no matter how advanced network technology becomes, packet loss during transmission is unavoidable, leading to distortion of patients' historical vital signs data. Ensuring the integrity and real-time nature of this data is a pressing issue that many remote consultation institutions urgently need to address. Summary of the Invention

[0004] The purpose of this application is to provide a method, apparatus, and equipment for verifying and repairing patient vital signs data, in order to solve the problem that existing patient vital signs data may experience packet loss during transmission, resulting in distortion of historical patient vital signs data and inability to guarantee real-time performance.

[0005] In a first aspect, embodiments of this application provide a method for verifying and repairing patient vital sign data, the method comprising: Acquire patient vital signs data collected by the terminal; Obtain frame images from a streaming media server, wherein the frame images are video frames obtained by parsing the video stream obtained from the terminal; The content of the frame image is sampled to obtain image sampling data; Compare the frequency information of the image sampling data and the patient's vital signs data at the same time. When it is determined that there is data loss in the patient's vital signs data, the frequency information of the image sampling data is used to repair the patient's vital signs data.

[0006] In some possible embodiments, sampling the content of the frame image to obtain image sampling data includes: Determine the first sampling frequency for the patient's vital signs data; The second sampling frequency is determined based on the first sampling frequency; The content of the frame image is sampled using the second sampling frequency to obtain image sampling data.

[0007] In some possible embodiments, when it is determined that the patient's vital signs data has data loss, repairing the patient's vital signs data using the frequency information of the image sampling data includes: Based on the frequency differences of the frequency information at the same time, it is determined that there is data loss in the patient's vital signs data; Based on the frequency information of the image sampling data, the patient's vital signs data collected at that time are repaired using interpolation.

[0008] In some possible embodiments, comparing the frequency information of the image sampling data and the patient's vital signs data at the same time includes: In response to the wavelet decomposition instruction, the image sampling data is decomposed using the mother wavelet to obtain the first frequency information of the image sampling data at different times. The patient's vital signs data are decomposed using the mother wavelet to obtain the second frequency information corresponding to the patient's vital signs data at different times; Compare the first frequency information and the second frequency information at the same time; The mother wavelet is a waveform that conforms to the frequency variation characteristics of the patient's vital signs data.

[0009] In some possible embodiments, the wavelet decomposition is performed layer by layer according to a set number of layers. When it is determined that there is data loss in the patient's vital signs data, the patient's vital signs data is repaired using the frequency information of the image sampling data, including: When performing wavelet decomposition layer by layer, the frequency difference between the first frequency information and the second frequency information at the same time is compared layer by layer. The target time and target layer for the first occurrence of lost patient vital signs data are determined based on the frequency differences. The patient's vital signs data collected at the target time are repaired using the second frequency information of the target layer at the target time. After the patient vital signs data of the target layer are repaired, the wavelet decomposition instruction is triggered again until no patient vital signs data is lost in any layer.

[0010] In some possible embodiments, the step of performing wavelet decomposition layer by layer according to the set number of layers includes: According to the set number of layers, determine the first sampling point / second sampling point of the image sampling data / patient vital sign data in the current layer layer by layer; The mother wavelet is used to perform convolution operation on the first / second sampling point of the current layer; Based on the convolution scale and displacement corresponding to the maximum convolution value, determine the first wavelet coefficients and second wavelet coefficients in the frequency domain corresponding to the first sampling point and the second sampling point of the current layer; The first frequency information and the second frequency information of the current layer are determined based on the first wavelet coefficient and the second wavelet coefficient.

[0011] In some possible embodiments, determining that there is data loss in the patient's vital signs data based on the frequency differences of the frequency information at the same time includes: Determine the difference between the first and second wavelet coefficients of the current layer at the same time; If the absolute value of the difference is greater than a set threshold, it is determined that the patient's vital signs data collected at the time point are lost in the current layer.

[0012] In some possible embodiments, the patient vital signs data are heart rate data, the frame image is an electrocardiogram, and the mother wavelet is a SYM4 wavelet.

[0013] In some possible embodiments, the video stream is an H.265 encoded video stream.

[0014] Secondly, embodiments of this application provide a patient vital sign data verification and repair device, the device comprising: The feature data acquisition module is used to acquire patient vital sign data collected by the terminal. A frame image acquisition module is used to acquire frame images from a streaming media server. The frame images are video frames obtained by parsing the video stream acquired from the terminal. An image sampling module is used to sample the content of the frame image to obtain image sampling data; A frequency comparison module is used to compare the frequency information of the image sampling data and the patient's vital signs data at the same time. The data repair module is used to repair the patient's vital signs data by using the frequency information of the image sampling data when it is determined that there is data loss in the patient's vital signs data.

[0015] Thirdly, another embodiment of this application provides a patient vital signs data verification and repair device, including at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform any of the patient vital signs data verification and repair device methods provided in the embodiments of this application.

[0016] Fourthly, another embodiment of this application also provides a computer storage medium storing a computer program for causing a computer to execute any of the patient vital sign data verification and repair device methods provided in the embodiments of this application.

[0017] The patient vital sign data verification and repair method, apparatus, and equipment provided in this application not only ensure the real-time requirement of data during remote patient consultations but also effectively guarantee the integrity requirement of historically collected patient data. This solves the problem that existing patient vital sign data suffers packet loss during transmission, leading to distortion of historical patient vital sign data and a failure to guarantee real-time performance.

[0018] Other features and advantages of this application will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 A schematic diagram illustrating the application environment of the patient vital sign data verification and repair method provided in the embodiments of this application; Figure 2 This is a schematic diagram of an electrocardiogram provided in an embodiment of this application; Figure 3 This is a schematic diagram of a patient vital sign data verification and repair method provided in an embodiment of this application; Figure 4 This is a schematic diagram of the wavelet decomposition process provided in an embodiment of this application; Figure 5 Detailed flowchart of the patient vital sign data verification and repair method provided in the embodiments of this application; Figure 6 This is a structural diagram of the patient vital sign data verification and repair device provided in the embodiments of this application; Figure 7 A schematic diagram of a patient vital sign data verification and repair device provided in an embodiment of this application. Detailed Implementation

[0021] To further illustrate the technical solutions provided in the embodiments of this application, a detailed description is provided below in conjunction with the accompanying drawings and specific implementation methods. Although the embodiments of this application provide method operation steps as shown in the following embodiments or drawings, more or fewer operation steps may be included in the method based on conventional or non-inventive effort. For steps that do not logically have a necessary causal relationship, the execution order of these steps is not limited to the execution order provided in the embodiments of this application. In actual processing or when the control device executes the method, it may be executed sequentially or in parallel according to the method shown in the embodiments or drawings.

[0022] Ensuring the integrity and real-time nature of historical vital signs data generated during remote consultations is a pressing issue for many remote consultation institutions.

[0023] After in-depth investigation and research to address the above issues, the first consideration was to simultaneously transmit the parsed data over the network and store it locally on the acquisition terminal to achieve delayed calibration. While this method can ensure the integrity of the patient's historical vital signs data, it cannot guarantee the real-time requirements of remote consultations under unstable network conditions. The second consideration was to transmit the monitor screen display content in real time via video streaming. Video compression technology can ensure real-time data transmission under low bandwidth conditions, but this method presents significant challenges in extracting patient vital signs data, especially waveform data.

[0024] In view of the fact that packet loss may occur during the transmission of patient vital signs data in related technologies, resulting in the distortion of patient historical vital signs data and the inability to guarantee real-time performance, this application proposes a method, device and equipment for verifying and repairing patient vital signs data.

[0025] Other features and advantages of this application will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings.

[0026] The following describes in detail, with reference to the accompanying drawings, the method, apparatus and equipment for verifying and repairing patient vital signs data in the embodiments of this application.

[0027] See Figure 1This is a schematic diagram illustrating an application scenario of a patient vital sign data verification and repair method according to an embodiment of this application. The scenario includes a terminal, a gateway server, and a streaming media server. The terminal connects to a monitoring device via a network port to receive patient vital sign data sent by the monitoring device. Simultaneously, it connects to a video capture card within the monitoring device via an HDMI (High Definition Multimedia Interface) to acquire the video stream captured by the video capture card from the monitoring device screen. Specific video stream data is as follows: Figure 2 As shown, the terminal sends patient vital sign data to the gateway server. The gateway server then stores the parsed patient vital sign data in a database (DB) using a parsing driver. The streaming media server parses the video stream and sends the parsed frame images to the doctor's client for remote diagnosis. This technology allows doctors to view the patient's entire vital sign information through video recording, but it has the disadvantage of requiring significant disk space. Furthermore, its efficiency in extracting real-time patient vital signs is very low, which is not conducive to the needs of hospitals or research institutions to build big data models based on patient vital sign data.

[0028] In this embodiment of the application, the streaming media server also uploads the frame images to the gateway server in real time, and the gateway server verifies and repairs the patient's vital signs data in real time based on the frame images.

[0029] Remote consultations demand extremely high real-time data accuracy. For critically ill patients, data delays and gaps can significantly impact a doctor's assessment. While terminals can collect and transmit patient vital signs data in real-time via a local area network through the monitor's network port, network latency and jitter are unavoidable given that remote consultations transmit patient vital signs data over the internet. Research indicates that transmitting video stream data using H.265 encoding and decoding is a highly reliable solution.

[0030] The aforementioned video stream data may, but is not limited to, video streams encoded with H.265. H.265 employs a high-efficiency coding rate, enabling real-time playback even in low-bandwidth environments. Since H.265 uses NALUs (Network Abstraction Layer Units) as the basic data unit, each NALU contains a type identifier and a sequence number. When frame loss occurs, the receiving end can identify key frames through the NALU type (e.g., I-frame, P-frame, etc.) and request retransmission of the key frames or recover data through other mechanisms, effectively ensuring the real-time requirements of remote diagnosis and treatment. Based on the data reliability of the video stream received by the streaming media server, this application embodiment utilizes frame images provided by the streaming media server to verify and restore the integrity of patient vital sign data.

[0031] Real-time video streaming ensures that doctors can obtain the current physical indicators and waveforms of remote patients in real time. Doctors can also retrieve historical patient vital sign data from the database. This historical data ensures that doctors can access patients' historical vital sign information outside of treatment. The combination of these two methods fully guarantees the real-time nature of data during patient treatment and the integrity of data in non-treatment situations. However, since conventional methods for sending historical patient vital sign data to the hospital's gateway server via TCP (Transmission Control Protocol) inevitably encounter a series of network, server, and application-related problems such as network jitter, packet loss, server bandwidth limitations, server downtime, and application anomalies, leading to data loss, conventional methods cannot guarantee the integrity of historical patient vital sign data.

[0032] Considering that the aforementioned problems are unavoidable in daily operations, this application embodiment uses a dual-server approach to prevent data loss caused by the hospital gateway server itself or the hospital gateway application itself. The dual-server approach mentioned in this application embodiment is not a traditional dual-machine backup, as this method cannot solve network transmission problems such as packet loss and latency. Since both the gateway server and the streaming media server are deployed on the hospital's intranet, theoretically, these two servers should not have network transmission problems. In this application embodiment, while acquiring the video stream, the streaming media server sends the parsed frame images to the gateway server. The gateway server samples the content of the frame images and verifies the acquired patient vital sign data based on the image sampling data. If the verification passes, the patient vital sign data is saved to the database (DB); if the verification fails, the patient vital sign data is repaired using the image sampling data and then stored in the database.

[0033] like Figure 3 As shown in the embodiment of this application, a method for verifying and repairing patient vital signs data is provided. This method is applied to an in-hospital gateway server and includes: Step 301: Obtain patient vital sign data collected by the terminal; The aforementioned terminal collects different types of patient vital sign data from corresponding monitoring equipment. This patient vital sign data is discrete time-domain data. The monitoring equipment integrates a video capture card, which is used to collect video streams corresponding to the patient's characteristic data. On one hand, the corresponding monitoring equipment collects patient vital sign data and sends it to the terminal via the network port. On the other hand, the terminal obtains the collected video stream from the video capture card via HDMI and sends the acquired video stream to the hospital's streaming media server. The aforementioned video stream can be an H.265 encoded video stream. In the event of frame loss, data recovery is performed using relevant keyframe recovery methods.

[0034] Step 302: Obtain a frame image from the streaming media server. The frame image is a video frame obtained by parsing the video stream obtained from the terminal. The streaming media server and the gateway server are both part of the hospital's servers, which ensures the reliability of video stream transmission and supports the verification and repair of patient vital signs data using the content of frame images.

[0035] Step 303: Sample the content of the frame image to obtain image sampling data; The patient's vital signs data mentioned above are discrete data in the time domain. The content related to the patient's vital signs in the frame image mentioned above is a continuous signal. In order to verify and repair the patient's vital signs data, it is necessary to sample the content of the frame image to obtain discrete image sampling data in the time domain.

[0036] Step 304: Compare the frequency information of image sampling data and patient vital sign data at the same time. The patient vital signs data and image sampling data obtained above are discrete time-domain signals. In order to obtain the frequency information of the patient vital signs data and image sampling data at different times, wavelet decomposition can be used to process the image sampling data and patient vital signs data respectively to obtain the time-frequency information of the image sampling data and the time-frequency information of the patient vital signs data.

[0037] Step 305: When it is determined that there is data loss in the patient's vital signs data, the patient's vital signs data is repaired using the frequency information of the image sampling data.

[0038] This application proposes a data transmission method combining the two methods described above. The acquired patient vital sign data is verified in real-time using frame images. If packet loss or errors occur in the acquired patient vital sign data, wavelet transform is performed on the image sampling data from the same time period. The time-frequency data obtained after the wavelet transform is then used to restore the integrity of the acquired patient vital sign data. This method not only ensures the real-time requirement of data during remote patient consultations but also effectively guarantees the integrity of historically acquired patient data.

[0039] This application embodiment can meet the needs of using video streams to assist doctors in remote consultations and using patient vital sign data pushed by monitor devices to complete the review and modeling of patient vital sign data. While the terminal collects real-time patient vital sign data sent by the monitor device, it also collects the monitor's video stream data through a video capture card. Considering the reliability advantages of video stream data, the patient vital sign data pushed by the monitor is verified using frame images from the same time period. Data that passes verification is directly stored in the database, while data that fails verification undergoes wavelet transform on the frame images, and interpolation is performed to repair the data based on the transformed time-frequency data.

[0040] In some possible embodiments, sampling the content of a frame image to obtain image sampling data includes: determining a first sampling frequency for patient vital signs data; determining a second sampling frequency based on the first sampling frequency; and sampling the content of the frame image using the second sampling frequency to obtain image sampling data. The method for determining the second sampling frequency based on the first sampling frequency may be to multiply the first sampling frequency by a set coefficient, where the set coefficient is a positive number greater than 1.

[0041] Because the vital signs data preserved in the frame image are continuous signals, while the gateway server acquires discrete signals, taking the patient's vital signs data collected by the monitor as ECG data as an example, the ECG waveform in the frame image is a continuous waveform signal, while the ECG data acquired by the gateway is a discrete signal. Therefore, it is necessary to sample the ECG waveform on the frame image. According to the Nyquist-Shannon sampling theorem (image sampling frequency f...),... s ≥2f max ), f max The sampling frequency for discrete patient vital signs data is the highest frequency obtained after decomposing the ECG data acquired by the gateway. Substitute 2 into the sampling frequency to obtain the sampling data of the ECG waveform on the image.

[0042] In some possible embodiments, when it is determined that there is data loss in the patient's vital signs data, the patient's vital signs data is repaired using the frequency information of the image sampling data, including: determining that there is data loss in the patient's vital signs data based on the frequency difference of the frequency information at the same time; and repairing the patient's vital signs data collected at that time using an interpolation method based on the frequency information of the image sampling data.

[0043] If there is no data loss in the patient's vital signs data, the frequency information of the patient's vital signs data and the image sampling data at the same time is relatively close. If there is data loss in the patient's vital signs data, the frequency information at the same time will have a large difference. Based on the frequency information of the image sampling data, the missing frequency information of the patient's vital signs data can be determined. Based on the missing frequency information, the patient's vital signs data collected at that time can be repaired by interpolation.

[0044] In some possible embodiments, comparing the frequency information of image sampling data and patient vital signs data at the same time includes: responding to a wavelet decomposition instruction, performing wavelet decomposition on the image sampling data using a mother wavelet to obtain first frequency information of the image sampling data at different times; performing wavelet decomposition on the patient vital signs data using the mother wavelet to obtain second frequency information of the patient vital signs data at different times; comparing the first frequency information and the second frequency information at the same time; wherein the mother wavelet is a waveform that conforms to the frequency change characteristics of the patient vital signs data.

[0045] For different types of patient vital sign data, a waveform that matches the frequency variation characteristics of the patient vital sign data can be used as the mother wavelet. In some possible embodiments, the aforementioned patient vital sign data is electrocardiogram (ECG) data, the frame image is an ECG waveform image, and the mother wavelet is a SYM4 wavelet.

[0046] In this application embodiment, the gateway server has two roles: 1) The gateway server is the TCP server of the terminal, that is, the gateway server acts as the receiving end for the monitoring equipment to send patient vital sign data; 2) The gateway server is the client side of the streaming media server, that is, the gateway server acts as the terminal client that pushes the monitor screen image in real time through the video capture card.

[0047] In some possible embodiments, the above wavelet decomposition is performed layer by layer according to a set number of layers, including: For image sampling data, the first sampling point of the image sampling data in the current layer is determined layer by layer according to the set layer number; for example, such as Figure 4 As shown, the number of layers is set to 8, and the image sampling data X is 0~250. The sampling points of the first layer are the first 0~125 sampling points CA0, which are used for wavelet decomposition to obtain wavelet coefficients, which are approximation coefficients CA. The wavelet coefficients obtained by wavelet decomposition of the last 125~250 sampling points CD0 are detail coefficients. Then, the reconstruction function is used to obtain the approximation components CA1 and detail components CD1 at different levels. The second layer is used to obtain the first 0~62.5 sampling points of the approximation components of the previous layer, and so on up to the 8th layer. When performing wavelet transform at each layer, the mother wavelet is used to perform convolution operation on the first sampling point of the current layer; according to the convolution scale and shift corresponding to the maximum convolution value, the first wavelet coefficient corresponding to the first sampling point of the current layer in the frequency domain is determined; the first frequency information of the current layer is determined based on the first wavelet coefficient. Similarly, for patient vital signs data, the second sampling point of the patient vital signs data in the current layer is determined layer by layer according to the set number of layers; the mother wavelet is used to perform convolution operation on the second sampling point of the current layer; the second wavelet coefficient corresponding to the second sampling point of the current layer in the frequency domain is determined according to the convolution scale and displacement corresponding to the maximum convolution value; and the second frequency information of the current layer is determined according to the second wavelet coefficient.

[0048] In this embodiment, the first sampling point of the image sampling data in the current layer is determined layer by layer according to the set number of layers. Starting from the first layer, wavelet transform is performed layer by layer until the last layer. When performing wavelet transform in the current layer, the mother wavelet is first scaled by a scale 'a' and then translated by a displacement 'b'. The 'a' and 'b' corresponding to the maximum convolution value are the convolution scale and displacement of the current layer. After wavelet decomposition, CAx represents the approximate components at different levels, and CDx represents the detail components at different levels. Since this is for data verification, this embodiment only needs to consider the coefficients of CA to achieve the purpose of data verification.

[0049] When the patient's vital signs data are ECG data and the frame image is an ECG (Electro Cardio Gram) waveform image, the wavelet coefficients C(a,b) after wavelet transform are expressed as follows:

[0050] Where 'a' represents the convolution scale used in the wavelet transform convolution operation, 'b' represents the shift used in the wavelet transform convolution operation, and 'f(t)' represents the acquired patient ECG data or data sampled from the ECG waveform. The conjugate function of the mother wavelet is used to calculate the values ​​of C(a,b) at eight scales, which are the wavelet coefficients of each decomposition level, for both the patient's ECG data and the data sampled from the ECG waveform.

[0051] The above process of calculating wavelet coefficients can be implemented using Matlab. Specifically, the calculations can be performed using Matlab as follows: coeffs = wavedec(ECG Signal, 8, sym4); The wavelet decomposition function `wavedec` is used to perform eight levels of wavelet decomposition on ECG data or sampled ECG waveforms using the SYM4 mother wavelet. This yields the approximation coefficients and detail coefficients `coeffs` at each of the eight levels. Then, a reconstruction function is used to obtain the approximation components `CAx` and detail components `CDx` at different levels. The specific expressions are as follows: The first level is defined as firstLevelCoeffs = wrcoef('d', coeffs, sym4, 1), and the secondLevelCoeffs is defined as secondLevelCoeffs = wrcoef('d', coeffs, sym4, 2), etc.

[0052] In some possible implementations, wavelet decomposition is performed layer by layer according to the set number of layers. When it is determined that there is data loss in the patient's vital signs data, the frequency information of the image sampling data is used to repair the patient's vital signs data. This includes: determining the frequency difference between the first frequency information and the second frequency information determined at the same time during the layer-by-layer wavelet decomposition; determining the target time and target layer where the patient's vital signs data is first lost based on the frequency difference; repairing the patient's vital signs data collected at the target time using the second frequency information of the target layer at the target time; and after the patient's vital signs data repair is completed at the target layer, the wavelet decomposition instruction is triggered again until no patient's vital signs data loss occurs at any layer.

[0053] In some possible embodiments, determining that there is data loss in patient vital signs data based on the frequency difference of frequency information at the same time includes: determining the difference between the first wavelet coefficient and the second wavelet coefficient at the same time in the current layer; if the absolute value of the difference is greater than a set threshold, determining that there is data loss in the patient vital signs data collected at that time in the current layer.

[0054] The specific process of the patient vital sign data verification and repair method in this application embodiment is as follows: Figure 5 As shown, the specific process includes: Step 501: The terminal collects the patient's vital signs data through the monitoring equipment and transmits it to the hospital gateway server through the network port. At the same time, it collects video streams through the video capture card and transmits them to the hospital streaming media server. Step 502: The hospital gateway server receives patient vital sign data through the network port, and at the same time receives frame images obtained by parsing the video stream from the hospital streaming media server. Step 503a: The in-hospital gateway server performs 8-level wavelet decomposition on the received patient vital sign data using the SYM4 wavelet basis. Step 503b: The in-hospital gateway server performs 8-level wavelet decomposition on the image sampling data obtained from the frame image sampling using the SYM4 wavelet basis. Step 504: Compare the results of the first-level wavelet decomposition and determine the absolute value of the difference in wavelet coefficients at the same time. Step 505: Is the absolute value of the difference between the first layer wavelet coefficients greater than the set threshold? If yes, proceed to step 512; otherwise, proceed to step 506. Taking patient vital signs data as ECG data as an example, the ECG data and the data obtained by sampling the ECG waveform in the frame image are decomposed into 8-level decomposed wavelet coefficients during the ECG data transmission time period of the patient. If the absolute value of the Coeffs difference of the first level of the two sets of data is less than 0.05, then the ECG data transmitted by the patient can be considered as complete data and can be saved to the database normally.

[0055] Step 506: Compare the absolute values ​​of the wavelet coefficient differences at the same time for other layers; Step 507: Locate the level where the first wavelet coefficient error is not less than 0.05; Step 508: Obtain the missing frequency information of the ECG data acquired at this level by using the wavelet time-frequency information of the frame image at this level; Step 509: Repair the collected patient vital signs data using interpolation. Step 510: Perform 8-level wavelet decomposition on the repaired data again; Step 511: Determine whether the absolute value of the wavelet coefficient difference is greater than the set threshold. If yes, proceed to step 506; otherwise, return to step 512. Step 512: Store the patient's vital signs data into the database DB.

[0056] If the absolute value of the difference in the first layer is not less than 0.05, it indicates that there is a data gap in a certain layer of the ECG data transmitted by the patient. After obtaining the time-frequency information of the two sets of data through wavelet transform, the time-frequency information obtained from image sampling is used to check the missing frequencies in the time-frequency information of the ECG data transmitted by the patient. Then, the data is inserted into the corresponding time period of the ECG data transmitted by the patient through interpolation. The wavelet transform of 8 layers is performed again until the absolute value of the difference is below 0.05. The data is considered to be repaired and is saved to the database normally. Data interpolation involves data fine-tuning. In order to simplify the calculation, this application obtains the low-frequency information of the 8-layer decomposition and performs time-frequency checks layer by layer. If a gap is found, the layer is interpolated and decomposed again.

[0057] Taking electrocardiogram (ECG) data as an example, the patient's ECG waveform, acquired by the gateway and sent by the monitor via TCP, undergoes an 8-level wavelet decomposition. Since the SYM4 wavelet basis is similar to the QRS complex in the ECG, it is selected as the mother wavelet. After decomposition, the wavelet time-frequency information (frequency information at different times) and wavelet coefficients at each level are obtained. The frame image sent by the streaming media server is preprocessed, and the ECG portion of the frame image is selected for masking. Since the image retains the continuous ECG signal, while the gateway acquires the discrete signal of the ECG data, the ECG waveform on the frame image is sampled. According to the Nyquist-Shannon sampling theorem, the highest frequency after decomposition of the ECG acquired by the gateway is... Substitute 2 as the sampling frequency to obtain the sampled data of the ECG waveform on the image, such as... Figure 2As shown in the labeled area. The sampled data was also decomposed into 8 levels using SYM4 wavelets. Wavelet coefficients for each of the 8 levels were obtained. Each level's wavelet coefficient was compared with the wavelet coefficients of the ECG data. If the absolute value of the difference was less than 0.05, the ECG data transmission was considered normal; otherwise, the ECG data was considered missing. For missing data, the absolute values ​​of the differences between the remaining 7 levels of wavelet coefficients were checked sequentially. If the absolute value of the difference between any level's wavelet coefficients was not less than 0.05, that level was considered to have missing data. The time-frequency information for that level was calculated using the ECG data and the sampled ECG waveform data. Specifically, approximate components were reconstructed using the corresponding wavelet coefficients, and this approximate component was used as the time-frequency information for that level. The missing frequencies for that level were identified, and interpolation was used to restore the integrity of the missing frequencies in the ECG data.

[0058] This embodiment of the application performs wavelet analysis on the real-time patient vital sign information sent by the monitoring device and the video information acquired by the monitoring device through the video capture card. The video acquisition, using H.265 encoding and decoding, ensures real-time video stream transmission with low bandwidth. Keyframes can be identified through NALU types (I-frame, P-frame, etc.), and retransmission of keyframes can be requested or data can be recovered through other mechanisms. Based on these properties, the patient's vital sign information in the frame image can be used as a reference for the real-time patient vital sign information sent by the monitoring device. Since the patient's vital sign data is discrete data and the frame image data is continuous data, it is difficult to analyze the values ​​of missing data points through time-domain analysis; only the time period of abnormality can be identified. Therefore, this embodiment of the application samples the frame image data and then performs SYM4 8-layer wavelet decomposition. Simultaneously, the patient's vital sign data sent by the monitoring device undergoes the same wavelet decomposition. By comparing and analyzing the time-frequency domain information obtained after wavelet decomposition, it verifies whether the patient's vital sign information is missing. If so, the missing frequencies are inserted into the transmitted data using interpolation to repair the data.

[0059] This application embodiment changes the traditional method of central stations relying solely on patient vital sign data pushed by monitors for real-time display. It acquires real-time video information from monitors through a video capture card, and ensures the real-time nature of remote consultations through H265 encoding and decoding. At the same time, it verifies the patient vital sign data sent by the monitors and repairs abnormal data, thus ensuring the integrity of historical data.

[0060] This application utilizes wavelet bandpass filtering features to compare patient vital sign data obtained through hierarchical decomposition with vital sign data sampled from video images, thereby determining the completeness of the collected patient vital sign data. By leveraging wavelet time-frequency analysis features, in cases where the collected patient vital sign data is incomplete, time-frequency analysis is used to determine the missing wavelet frequency information and its corresponding time period. The missing frequency information is then used to insert the missing data into the abnormally collected patient vital sign data using interpolation, thus restoring the integrity of the missing data.

[0061] This application embodiment uses complete historical patient vital signs data for a review of vital signs, which not only allows doctors to view complete vital signs information during the treatment period, but also solves the problem of large hard disk space usage caused by video storage.

[0062] Based on the same inventive concept, this application also provides a patient vital sign data verification and repair device, such as... Figure 6 As shown, the device includes: The feature data acquisition module 601 is used to acquire patient vital sign data collected by the terminal. The frame image acquisition module 602 is used to acquire frame images from the streaming media server, wherein the frame images are video frames obtained by performing frame parsing on the video stream acquired from the terminal. Image sampling module 603 is used to sample the content of the frame image to obtain image sampling data; The frequency comparison module 604 is used to compare the frequency information of the image sampling data and the patient's vital signs data at the same time. The data repair module 605 is used to repair the patient's vital signs data by using the frequency information of the image sampling data when it is determined that there is data loss in the patient's vital signs data.

[0063] In some possible embodiments, the image sampling module samples the content of the frame image to obtain image sampling data, including: Determine the first sampling frequency for the patient's vital signs data; The second sampling frequency is determined based on the first sampling frequency; The content of the frame image is sampled using the second sampling frequency to obtain image sampling data.

[0064] In some possible embodiments, when the data repair module determines that there is data loss in the patient's vital signs data, it repairs the patient's vital signs data using the frequency information of the image sampling data, including: Based on the frequency differences of the frequency information at the same time, it is determined that there is data loss in the patient's vital signs data; Based on the frequency information of the image sampling data, the patient's vital signs data collected at that time are repaired using interpolation.

[0065] In some possible embodiments, the frequency comparison module compares the frequency information of the image sampling data and the patient's vital signs data at the same time, including: In response to the wavelet decomposition instruction, the image sampling data is decomposed using the mother wavelet to obtain the first frequency information of the image sampling data at different times. The patient's vital signs data are decomposed using the mother wavelet to obtain the second frequency information corresponding to the patient's vital signs data at different times; Compare the first frequency information and the second frequency information at the same time; The mother wavelet is a waveform that conforms to the frequency variation characteristics of the patient's vital signs data.

[0066] In some possible embodiments, the wavelet decomposition is performed layer by layer according to a set number of layers. When the frequency comparison module determines that there is data loss in the patient's vital signs data, it repairs the patient's vital signs data using the frequency information of the image sampling data, including: When performing wavelet decomposition layer by layer, the frequency difference between the first frequency information and the second frequency information at the same time is compared layer by layer. The target time and target layer for the first occurrence of lost patient vital signs data are determined based on the frequency differences. The patient's vital signs data collected at the target time are repaired using the second frequency information of the target layer at the target time. After the patient vital signs data of the target layer are repaired, the wavelet decomposition instruction is triggered again until no patient vital signs data is lost in any layer.

[0067] In some possible embodiments, the frequency comparison module performs wavelet decomposition layer by layer according to a set number of layers, including: According to the set number of layers, determine the first sampling point / second sampling point of the image sampling data / patient vital sign data in the current layer layer by layer; The mother wavelet is used to perform convolution operation on the first / second sampling point of the current layer; Based on the convolution scale and displacement corresponding to the maximum convolution value, determine the first wavelet coefficients and second wavelet coefficients in the frequency domain corresponding to the first sampling point and the second sampling point of the current layer; The first frequency information and the second frequency information of the current layer are determined based on the first wavelet coefficient and the second wavelet coefficient.

[0068] In some possible embodiments, the image sampling module determines that there is data loss in the patient's vital signs data based on the frequency differences of the frequency information at the same time, including: Determine the difference between the first and second wavelet coefficients of the current layer at the same time; If the absolute value of the difference is greater than a set threshold, it is determined that the patient's vital signs data collected at the time point are lost in the current layer.

[0069] In some possible embodiments, the patient's vital signs data are electrocardiogram (ECG) data, the frame image is an ECG waveform image, and the mother wavelet is a SYM4 wavelet.

[0070] In some possible embodiments, the video stream is an H.265 encoded video stream.

[0071] Having introduced the patient vital signs data verification and repair method and apparatus according to exemplary embodiments of this application, we will now introduce a patient vital signs data verification and repair device according to another exemplary embodiment of this application.

[0072] Those skilled in the art will understand that various aspects of this application can be implemented as a system, method, or program product. Therefore, various aspects of this application can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, collectively referred to herein as a "circuit," "module," or "system."

[0073] In some possible implementations, the patient vital signs data verification and repair device according to this application may include at least one processor and at least one memory. The memory stores program code that, when executed by the processor, causes the processor to perform the steps of the patient vital signs data verification and repair method according to the various exemplary embodiments of this application described above.

[0074] The following reference Figure 7 This application describes a patient vital signs data verification and repair device 170 according to one embodiment of the present application. Figure 7 The patient vital signs data verification and repair device 170 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0075] like Figure 7 As shown, the patient vital signs data verification and repair device 170 is presented in the form of a general electronic device. The components of the patient vital signs data verification and repair device 170 may include, but are not limited to: at least one processor 171, at least one memory 172, and a bus 173 connecting different system components (including memory 172 and processor 171).

[0076] Bus 173 represents one or more of several bus architectures, including a memory bus or memory controller, peripheral bus, processor, or local bus using any of the various bus architectures.

[0077] The memory 172 may include a readable medium in the form of volatile memory, such as random access memory (RAM) 1721 and / or cache memory 1722, and may further include read-only memory (ROM) 1723.

[0078] The memory 172 may also include a program / utility 1725 having a set (at least one) of program modules 1724, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0079] The patient vital signs data verification and repair device 170 can also communicate with one or more external devices 174 (e.g., keyboard, pointing device, etc.), one or more devices that enable a user to interact with the patient vital signs data verification and repair device 170, and / or any device that enables the patient vital signs data verification and repair device 170 to communicate with one or more other electronic devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 175. Furthermore, the patient vital signs data verification and repair device 170 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 176. As shown, network adapter 176 communicates with other modules used for the patient vital signs data verification and repair device 170 via bus 173. It should be understood that, although not shown in the figure, other hardware and / or software modules, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, can be used in conjunction with patient vital signs data verification and repair 170.

[0080] In some possible implementations, various aspects of the patient vital signs data verification and repair method provided in this application can also be implemented in the form of a program product, which includes program code. When the program product is run on a computer device, the program code is used to cause the computer device to perform the steps in the patient vital signs data verification and repair method according to various exemplary embodiments of this application described above.

[0081] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0082] The program product for verifying and repairing patient vital signs data according to the embodiments of this application can be a portable compact disc read-only memory (CD-ROM) and include program code, and can run on an electronic device. However, the program product of this application is not limited thereto. In this document, the readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or apparatus.

[0083] A readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying readable program code. This propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0084] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0085] Program code for performing the operations of this application can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's electronic device, partially on the user's device, as a standalone software package, partially on the user's electronic device and partially on a remote electronic device, or entirely on a remote electronic device or server. In cases involving remote electronic devices, the remote electronic device can be connected to the user's electronic device via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external electronic device (e.g., via the Internet using an Internet service provider).

[0086] It should be noted that although several units or sub-units of the device have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of this application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided and embodied by multiple units.

[0087] Furthermore, although the operations of the method of this application are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0088] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0089] This application is described with reference to flowchart illustrations and block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block and / or segment of the flowchart illustrations and block diagrams, as well as combinations of blocks and segments in the flowchart illustrations and block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart. Figure 1 One or more processes and boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0090] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and boxes Figure 1 The function specified in one or more boxes.

[0091] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and boxes Figure 1 The steps of the function specified in one or more boxes.

[0092] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0093] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for verifying and repairing patient vital signs data, characterized in that, The method includes: Acquire patient vital signs data collected by the terminal; Obtain frame images from a streaming media server, wherein the frame images are video frames obtained by parsing the video stream obtained from the terminal; The content of the frame image is sampled to obtain image sampling data; Compare the frequency information of the image sampling data and the patient's vital signs data at the same time. When it is determined that there is data loss in the patient's vital signs data, the frequency information of the image sampling data is used to repair the patient's vital signs data.

2. The method according to claim 1, characterized in that, The step of sampling the content of the frame image to obtain image sampling data includes: Determine the first sampling frequency for the patient's vital signs data; The second sampling frequency is determined based on the first sampling frequency; The content of the frame image is sampled using the second sampling frequency to obtain image sampling data.

3. The method according to claim 1, characterized in that, When it is determined that the patient's vital signs data has data loss, the patient's vital signs data is repaired using the frequency information of the image sampling data, including: Based on the frequency differences of the frequency information at the same time, it is determined that there is data loss in the patient's vital signs data; Based on the frequency information of the image sampling data, the patient's vital signs data collected at that time are repaired using interpolation.

4. The method according to any one of claims 1 to 3, characterized in that, The frequency information of comparing the image sampling data and the patient's vital signs data at the same time includes: In response to the wavelet decomposition instruction, the image sampling data is decomposed using the mother wavelet to obtain the first frequency information of the image sampling data at different times. The patient's vital signs data are decomposed using the mother wavelet to obtain the second frequency information corresponding to the patient's vital signs data at different times; Compare the first frequency information and the second frequency information at the same time; The mother wavelet is a waveform that conforms to the frequency variation characteristics of the patient's vital signs data.

5. The method according to claim 4, characterized in that, The wavelet decomposition is performed layer by layer according to the set number of layers. When it is determined that there is data loss in the patient's vital signs data, the frequency information of the image sampling data is used to repair the patient's vital signs data, including: When performing wavelet decomposition layer by layer, the frequency difference between the first frequency information and the second frequency information at the same time is compared layer by layer. The target time and target layer for the first occurrence of lost patient vital signs data are determined based on the frequency differences. The patient's vital signs data collected at the target time are repaired using the second frequency information of the target layer at the target time. After the patient vital signs data of the target layer are repaired, the wavelet decomposition instruction is triggered again until no patient vital signs data is lost in any layer.

6. The method according to claim 5, characterized in that, The step of performing wavelet decomposition layer by layer according to the set number of layers includes: According to the set number of layers, determine the first sampling point / second sampling point of the image sampling data / patient vital sign data in the current layer layer by layer; The mother wavelet is used to perform convolution operation on the first / second sampling point of the current layer; Based on the convolution scale and displacement corresponding to the maximum convolution value, determine the first wavelet coefficients and second wavelet coefficients in the frequency domain corresponding to the first sampling point and the second sampling point of the current layer; The first frequency information and the second frequency information of the current layer are determined based on the first wavelet coefficient and the second wavelet coefficient.

7. The method according to claim 6, characterized in that, The step of determining whether the patient's vital signs data has data loss based on the frequency difference of the frequency information at the same time includes: Determine the difference between the first and second wavelet coefficients of the current layer at the same time; If the absolute value of the difference is greater than a set threshold, it is determined that the patient's vital signs data collected at the time point are lost in the current layer.

8. The method according to claim 3, characterized in that, The patient's vital signs data are electrocardiogram (ECG) data, the frame images are ECG waveform images, and the mother wavelet is the SYM4 wavelet.

9. A device for verifying and repairing patient vital signs data, characterized in that, The device includes: The feature data acquisition module is used to acquire patient vital sign data collected by the terminal. A frame image acquisition module is used to acquire frame images from a streaming media server. The frame images are video frames obtained by parsing the video stream acquired from the terminal. An image sampling module is used to sample the content of the frame image to obtain image sampling data; A frequency comparison module is used to compare the frequency information of the image sampling data and the patient's vital signs data at the same time. The data repair module is used to repair the patient's vital signs data by using the frequency information of the image sampling data when it is determined that there is data loss in the patient's vital signs data.

10. A device for verifying and repairing patient vital signs data, characterized in that, The method includes at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method as described in any one of claims 1-8.

Images