Pain monitoring system and method for lung cancer patient

Abstract

The invention provides a pain monitoring system and method for a lung cancer patient, and belongs to the technical field of image recognition and medical monitoring. The method comprises the following steps: acquiring a plurality of continuous video frames containing the posture of a lung cancer patient according to a preset period; determining a pain monitoring target frame in the continuous video frames; sending the pain monitoring target frame to an AI monitoring engine; the AI monitoring engine judges whether the patient suffering from the lung cancer feels painful or not based on the incidence relation between the facial expression contained in the pain monitoring target frame and the pain degree; when it is judged that the lung cancer patient feels pain, the pain monitoring target frame is marked as a responsibility frame; otherwise, marking the pain monitoring target frame as an interference frame. The inter-frame difference is a limb posture inter-frame difference. The system includes an AI monitoring engine for implementing the method. According to the technical scheme, the pain monitoring process is divided into a limb inter-frame difference filtering stage and an expression recognition and confirmation stage, and the accuracy of pain monitoring of the patient with the lung cancer is improved while the data processing complexity of each stage is reduced.

Description

Technical Field

[0001] The present invention belongs to the field of image recognition and medical monitoring technology, and in particular relates to a pain monitoring system and method for lung cancer patients, a computer-readable storage medium for implementing the method, a computer program product, and an electronic device. Background Art

[0002] Pain is a characteristic of many diseases and can be used as an important indicator to monitor the condition and measure the treatment effect in clinical practice. With the evaluation of graded hospitals, the evaluation of standardized cancer pain treatment demonstration wards, and the widespread practice of the concept of Enhanced Recovery After Surgery (ERAS) in the field of surgery, pain management has received more and more attention. Many evaluation standards and studies at home and abroad focus on the quality management of pain. For some special patients, such as lung cancer patients, efficient and accurate pain assessment results have more important clinical significance for the treatment and care of patients.

[0003] As we all know, human facial expressions contain a lot of pain information and are one of the common evaluation criteria for pain. For example, the literature (Pedersen H. Learning Appearance Features for Pain Detection Using the UNBC-McMaster Shoulder Pain Expression Archive Database [J]. Springer International Publishing, 2015.) proposed using the support vector machine (SVM) algorithm for pain recognition; Chinese invention patent applications such as publication numbers CN117334337A and CN117976217A all proposed pain assessment or recognition grading and classification methods using frame images and audio information.

[0004] However, on the one hand, the related technologies all perform pain recognition on specific priority images or audio frame samples of given recognition targets, and the number of samples to be processed is limited. In the actual monitoring scenario of lung cancer patients, it is impossible to predict when lung cancer patients will experience pain symptoms, so pain monitoring needs to be carried out continuously, which will generate a large number of samples to be recognized, and most of these samples to be recognized are actually normal samples (without pain symptoms). If frame-by-frame recognition is performed according to the method given in the related technology, it will obviously lead to a sharp increase in data processing volume while the effect is half the result with twice the effort. On the other hand, facial expression recognition requires accurate capture of facial expression changes, and in most normal situations (without pain symptoms), the patient's facial expression will not actually change much. If the existing technology is still used to perform frame-by-frame facial recognition or other features (such as audio features) capture and recognize, the complexity of the data processing process will also increase.

[0005] Therefore, the promotion and application of existing technologies in actual pain monitoring scenarios for lung cancer patients still has certain defects. At least there is still room for improvement in terms of reducing the complexity of data processing while taking into account accuracy and real-time performance. Summary of the invention

[0006] In view of the above technical problems, the present invention proposes a pain monitoring system and method for lung cancer patients, a computer-readable storage medium, a computer program product and an electronic device for implementing the method.

[0007] In a first aspect of the present invention, a pain monitoring method for lung cancer patients is proposed. The method is implemented based on an edge image acquisition device, and the image acquisition device is connected to an AI monitoring engine. The method comprises the following steps:

[0008] S110: Acquire a plurality of continuous video frames including the posture of the lung cancer patient according to a preset period;

[0009] S120: Determine a pain monitoring target frame in the continuous video frames;

[0010] S130: Sending the pain monitoring target frame to the AI ​​monitoring engine;

[0011] S140: The AI ​​monitoring engine analyzes the pain monitoring target frame to determine whether the lung cancer patient experiences pain;

[0012] When it is determined that the lung cancer patient has pain, marking the pain monitoring target frame as a responsibility frame and saving it in a responsibility frame database;

[0013] Otherwise, marking the pain monitoring target frame as an interference frame and saving it to the interference frame database;

[0014] Wherein, the step S120 includes:

[0015] Based on the responsible frame, the interference frame and the frame difference between a plurality of consecutive video frames, a pain monitoring target frame in the consecutive video frames is determined.

[0016] The lung cancer patient's posture in step S110 includes facial posture and limb posture; the facial posture includes facial expression posture, and the limb posture includes limb movements.

[0017] The step S120 further includes:

[0018] If the frame difference between two consecutive video frames Fa and Fb in the consecutive video frames is less than a preset frame difference threshold, the video frames Fa and Fb are not used as the pain monitoring target frames.

[0019] If the inter-frame difference between two consecutive video frames Fa and Fb in the consecutive video frames is greater than a preset inter-frame difference threshold, then determining the first inter-frame similarity between the video frame Fa or the video frame Fb and any of the responsible frames;

[0020] When the similarity between any of the first frames is greater than a first preset similarity threshold, both the video frame Fa and the video frame Fb are used as the pain monitoring target frames.

[0021] If the inter-frame difference between two consecutive video frames Fa and Fb in the consecutive video frames is greater than a preset inter-frame difference threshold, determining a second inter-frame similarity between the video frame Fa or the video frame Fb and any of the interference frames;

[0022] When any of the second inter-frame similarities is greater than a second preset similarity threshold, the video frame Fa and the video frame Fb are not used as the pain monitoring target frames.

[0023] The AI ​​monitoring engine analyzes the pain monitoring target frame to determine whether the lung cancer patient experiences pain, specifically including:

[0024] The AI ​​monitoring engine determines whether the lung cancer patient is experiencing pain based on the correlation between the facial expression contained in the pain monitoring target frame and the pain level.

[0025] In a second aspect of the present invention, in order to implement the pain monitoring method for lung cancer patients described in the first aspect, a pain monitoring system for lung cancer patients is proposed, the system comprising an AI monitoring engine;

[0026] The system further comprises:

[0027] A video frame acquisition unit, configured to acquire a plurality of continuous video frames including the posture of the lung cancer patient according to a preset period; the posture of the lung cancer patient includes the posture of limbs; and the posture of limbs includes the movements of limbs;

[0028] a pain monitoring target frame filtering unit, configured to determine a pain monitoring target frame in the continuous video frames based on a responsible frame, an interference frame, and an inter-frame difference between a plurality of continuous video frames, and send the pain monitoring target frame to the AI ​​monitoring engine;

[0029] The inter-frame difference is the inter-frame difference of the body posture of two consecutive video frames;

[0030] The AI ​​monitoring engine analyzes the pain monitoring target frame to determine whether the lung cancer patient experiences pain;

[0031] When it is determined that the lung cancer patient has pain, marking the pain monitoring target frame as a responsible frame;

[0032] Otherwise, the pain monitoring target frame is marked as an interference frame.

[0033] The lung cancer patient's posture also includes a facial posture; the facial posture includes a facial expression posture;

[0034] The AI ​​monitoring engine analyzes the pain monitoring target frame to determine whether the lung cancer patient experiences pain, specifically including:

[0035] The AI ​​monitoring engine determines whether the lung cancer patient is experiencing pain based on the correlation between the facial expression contained in the pain monitoring target frame and the pain level.

[0036] The pain monitoring target frame filtering unit determines the pain monitoring target frame in the continuous video frames based on the responsible frame, the interference frame and the frame difference between the multiple continuous video frames, specifically including:

[0037] If the frame difference between two consecutive video frames Fa and Fb in the consecutive video frames is less than a preset frame difference threshold, then the video frames Fa and Fb are not used as the pain monitoring target frames;

[0038] In one scenario, if the inter-frame difference between two consecutive video frames Fa and Fb in the consecutive video frames is greater than a preset inter-frame difference threshold, then determining the first inter-frame similarity between the video frame Fa or the video frame Fb and any of the responsible frames;

[0039] When the similarity between any of the first frames is greater than a first preset similarity threshold, both the video frame Fa and the video frame Fb are used as the pain monitoring target frames;

[0040] In another scenario, if the inter-frame difference between two consecutive video frames Fa and Fb in the consecutive video frames is greater than a preset inter-frame difference threshold, then determining the second inter-frame similarity between the video frame Fa or the video frame Fb and any of the interference frames;

[0041] When any of the second inter-frame similarities is greater than a second preset similarity threshold, the video frame Fa and the video frame Fb are not used as the pain monitoring target frames.

[0042] Part or all of the steps of a pain monitoring method for lung cancer patients described in the first aspect can be automatically implemented through various forms of electronic devices and computer program instructions; the computer program instructions can be stored in storage media of different forms and loaded into computer electronic devices for execution.

[0043] Therefore, in the third aspect of the present invention, a computer-readable storage medium is also provided for storing computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the pain monitoring method for lung cancer patients described in the first aspect.

[0044] In the fourth aspect of the present invention, an electronic device is also proposed, which includes a processor and a memory, the memory is used to store instructions, and the processor is used to call the instructions in the memory, so that the electronic device executes the pain monitoring method for lung cancer patients described in the first aspect above.

[0045] In a fifth aspect of the present invention, a computer program product is also proposed, the product comprising a computer program, and when the computer program is executed, the pain monitoring method for lung cancer patients described in the first aspect is implemented.

[0046] The technical solution of the present invention divides the pain monitoring process into a limb frame difference filtering stage and an expression recognition confirmation stage, which reduces the complexity of data processing in each stage and improves the accuracy of pain monitoring for lung cancer patients.

[0047] Further advantages of the present invention will be further reflected in detail in the specific embodiments section in conjunction with the drawings of the specification. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0049] Figure 1 This is a schematic diagram of the main flow of a pain monitoring method for lung cancer patients according to an embodiment of the present invention.

[0050] Figure 2 yes Figure 1 Schematic diagram of the process of determining the pain monitoring target frame in the continuous video frames in the embodiment

[0051] Figure 3 yes Figure 1 Schematic diagram of the principle of calculating the limb frame difference between consecutive video frames in the embodiment

[0052] Figure 4 This is a schematic diagram of the functional modules of a pain monitoring system for lung cancer patients according to an embodiment of the present invention. DETAILED DESCRIPTION

[0053] First of all, it should be pointed out that the embodiments of the pain monitoring method for lung cancer patients mentioned in this section can be implemented through a computer program on an electronic device or system configured with a memory and a processor. The electronic device or system can be in the form of a physical machine, a virtual machine, a server, a cluster, or any combination thereof.

[0054] Preferably, the specific form of the electronic device may also be a human-computer interaction terminal, and the human-computer interaction terminal may be a desktop terminal with a human-computer interaction interface, a smart handheld terminal, a mobile terminal, a medical handheld PDA, etc.

[0055] See first Figure 1 , Figure 1 It is a main flow chart of a pain monitoring method for lung cancer patients according to an embodiment of the present invention.

[0056] Figure 1 The method flow includes steps S110-S140 (step numbers are omitted in the drawings), and each step is specifically implemented as follows:

[0057] S110: Acquire a plurality of continuous video frames including the posture of the lung cancer patient according to a preset period;

[0058] S120: Determine a pain monitoring target frame in the continuous video frames;

[0059] S130: Sending the pain monitoring target frame to the AI ​​monitoring engine;

[0060] S140: The AI ​​monitoring engine analyzes the pain monitoring target frame to determine whether the lung cancer patient experiences pain;

[0061] When it is determined that the lung cancer patient has pain, marking the pain monitoring target frame as a responsible frame;

[0062] Otherwise, marking the pain monitoring target frame as an interference frame;

[0063] Wherein, the step S120 includes:

[0064] Based on the responsible frame, the interference frame and the frame difference between a plurality of consecutive video frames, a pain monitoring target frame in the consecutive video frames is determined.

[0065] When the method is specifically implemented, the method can be implemented based on an edge image acquisition device, and the edge image acquisition device is connected to an AI monitoring engine.

[0066] Preferably, the edge image acquisition device is a type of edge device, which is arranged proximal to the target lung cancer patient, and the edge device has basic image frame acquisition function, frame difference calculation function and similarity calculation function.

[0067] Preferably, the edge image acquisition device includes an image frame acquisition unit, an inter-frame difference calculation unit, a similarity calculation unit and a target frame filtering unit;

[0068] The image frame acquisition unit executes step S110; that is, the image frame acquisition unit acquires a plurality of continuous video frames including the posture of the lung cancer patient according to a preset period.

[0069] The lung cancer patient's posture includes facial posture and limb posture; the facial posture includes facial expression posture, and the limb posture includes limb movements.

[0070] Next, see Figure 2-Figure 3 , further introduction Figure 1 Specific implementation methods of other steps of the method embodiment.

[0071] On the basis of obtaining a plurality of continuous video frames including the posture of the lung cancer patient in step S110, an inter-frame difference calculation unit, a similarity calculation unit and a target frame filtering unit are used to perform step S120.

[0072] For details, see Figure 2 and Figure 3 .

[0073] Figure 2 A plurality of consecutive video frames containing the posture of the lung cancer patient are shown in FIG.

[0074] {Video frame 1, video frame 2, ..., video frame Fa, video frame Fb, ..., video frame N}. This embodiment is introduced by taking any two consecutive video frames Fa and Fb as an example.

[0075] The frame difference calculation unit calculates the frame difference between two consecutive video frames Fa, Fb.

[0076] The traditional frame difference calculation method needs to calculate the frame difference of all regions of the video frames Fa and Fb. In this embodiment, by consulting relevant literature, it can be known that the monitoring video frames of lung cancer patients are normal in most cases. Pain is more likely to occur only when there are changes in limb movements, especially back pain, chest pain, and abdominal pain. For example, "Health Preservation Classified Compilation" says: "Coughing and spitting, spitting does not need to be far away, or lung disease makes people feel heavy in the hands and feet and cough with back pain." The lung disease discussed here is more likely to be lung cancer, which then causes back pain.

[0077] Based on the above findings, in the inter-frame difference calculation unit of the present embodiment, the inter-frame difference calculation unit specifically calculates the inter-frame limb posture difference Fab of two consecutive video frames Fa and Fb;

[0078] For example, the region of the video frames Fa and Fb that only contains body gestures is selected for frame difference calculation. Preferably, the region that only contains body gestures excludes the head region in the video frames Fa and Fb, thereby reducing the recognition of micro-expressions.

[0079] If the inter-frame limb posture difference Fab between two consecutive video frames Fa and Fb in the continuous video frames is less than the preset inter-frame difference threshold Fset, then the video frames Fa and Fb are not used as the pain monitoring target frames.

[0080] In an actual monitoring scenario, if the inter-frame limb posture difference Fab of two consecutive video frames Fa and Fb in the continuous video frames is less than the preset inter-frame difference threshold Fset, that is, the limb posture has no change or obvious change (for example, no touching the head / chest, touching the back, etc.), it means that the current situation is most likely normal and no pain monitoring is required. Figure 2 It is shown in the figure that the target frame filtering unit filters out the continuous video frames Fa and Fb at this time, that is, there is no need to perform subsequent pain recognition on the continuous video frames Fa and Fb;

[0081] If the inter-frame limb posture difference Fab of two consecutive video frames Fa and Fb in the continuous video frames is greater than the preset inter-frame difference threshold Fset, it means that there is a significant change in the limb posture, which may mean that the current patient has pain symptoms and needs further identification.

[0082] Specifically, if the inter-frame limb posture difference between two consecutive video frames Fa and Fb in the consecutive video frames is greater than a preset inter-frame difference threshold,

[0083] Then the similarity calculation unit calculates the first inter-frame similarity Fs1 between the video frame Fa or the video frame Fb and any of the responsible frames;

[0084] When the present embodiment is actually implemented, a basic responsible frame database and an interference frame database may be pre-established. As the present method is gradually executed, the responsible frame database and the interference frame database implement self-learning feedback updates (step S140).

[0085] In the initial state, the basic responsibility frame database is loaded with a certain number of manually annotated limb movement frames that are determined to be related to the pain symptoms of lung cancer patients; the basic interference frame database is loaded with limb movement frames that are determined to be unrelated to the pain symptoms of lung cancer patients.

[0086] Thus, the similarity calculation unit can calculate a plurality of first inter-frame similarities Fs1 between the video frame Fa or the video frame Fb and any of the responsible frames;

[0087] When any of the first inter-frame similarities Fs1 is greater than the first preset similarity threshold Fset1, both the video frame Fa and the video frame Fb are used as the pain monitoring target frames, otherwise, neither the video frames Fa nor Fb are used as the pain monitoring target frames, that is, the target frame filtering unit filters out the continuous video frames Fa and Fb at this time, that is, there is no need to perform subsequent pain recognition on the continuous video frames Fa and Fb;

[0088] In another aspect, if the inter-frame difference between two consecutive video frames Fa and Fb in the consecutive video frames is greater than a preset inter-frame difference threshold, then determining a plurality of second inter-frame similarities between the video frame Fa or the video frame Fb and any of the interfering frames;

[0089] When any of the second inter-frame similarities is greater than a second preset similarity threshold, the video frame Fa and the video frame Fb are not used as the pain monitoring target frames.

[0090] Figure 3 yes Figure 1 A schematic diagram of the principle of calculating the limb frame differences of consecutive video frames in the embodiment.

[0091] Figure 3 Five consecutive video frames F1-F5 are shown. It can be seen that there are continuous and obvious limb changes in video frames F1-F4, and the limb differences between frames F12, F23, and F34 are very significant (greater than the threshold), while the changes between video frames F4-F5 are not very obvious, and the limb differences between frames (limb posture frame differences) are not obvious (less than the threshold).

[0092] Based on the above improvements of this embodiment, the technical solution of the present application divides the pain monitoring process into a limb frame difference filtering stage and an expression recognition confirmation stage when performing pain recognition.

[0093] The above-mentioned limb frame difference filtering stage can filter out the responsible frames that may contain pain symptoms based on the multiple video frames originally captured to enter the subsequent expression recognition confirmation stage. Compared with the defects of the prior art that frame-by-frame recognition leads to a sharp increase in data processing volume while achieving half the result with twice the effort, this embodiment can significantly reduce the data processing volume (only Figure 3 The example shown has already reduced the amount of data by 20%. Considering other scenarios and the increase in the base sample size, the reduction in data volume will be more significant).

[0094] At the same time, the above process is based on the calculation method of the frame difference of limb posture, which can significantly reduce the amount of graphics data processing. Because when the limb posture changes, the anchor calculation area changes significantly, and there is no need for high-resolution contour recognition and pixel segmentation.

[0095] At this point, the method is completed to step S130, that is, a small number of pain monitoring target frames are identified and sent to the AI ​​monitoring engine.

[0096] The AI ​​monitoring engine analyzes the pain monitoring target frame to determine whether the lung cancer patient experiences pain.

[0097] Specifically, the AI ​​monitoring engine analyzes the pain monitoring target frame to determine whether the lung cancer patient experiences pain, specifically including:

[0098] The AI ​​monitoring engine determines whether the lung cancer patient is experiencing pain based on the correlation between the facial expression contained in the pain monitoring target frame and the pain level.

[0099] The AI ​​monitoring engine can classify lung cancer patients into two levels of pain:

[0100] Level 1: Determine whether pain occurs;

[0101] Level 2: Determine the level of pain.

[0102] As introduced in the background technology, there are many models or algorithms in the prior art for specifically identifying and monitoring pain and classifying the levels. The present invention does not elaborate on this. The focus of the technical solution of the present invention is not on judging whether pain occurs or determining the level of pain, but on dividing the pain monitoring process into a limb frame difference filtering stage and an expression recognition confirmation stage to reduce the complexity of data processing in each stage.

[0103] Preferably, the AI ​​monitoring engine includes an SVM vector machine pain recognition classification model, a Bayesian network-based pain assessment model, etc., all of which can be found in relevant existing technologies.

[0104] It can be seen that in the expression recognition confirmation stage, the pain monitoring target frame to be processed is already a responsible frame with a high probability that has been filtered on the original monitoring video frame. At this time, the data processing volume has been greatly reduced.

[0105] However, the present invention has also been further improved on this basis as follows:

[0106] When it is determined that the lung cancer patient feels pain, the pain monitoring target frame is marked as a responsibility frame; otherwise, the pain monitoring target frame is marked as an interference frame.

[0107] As mentioned above, when actually implementing this embodiment, a basic responsible frame database and an interference frame database may be pre-established. As this method is gradually executed, the responsible frame database and the interference frame database realize self-learning feedback updates, namely the above-mentioned marking process.

[0108] According to the above-mentioned marking process, when applying the relevant AI monitoring engine, the responsible frame database and the interference frame database can be further updated based on the results of the AI ​​monitoring engine, thereby realizing the full-process closed-loop feedback self-learning update process of the method, so that the image corpus of the responsible frame database and the interference frame database is automatically enriched at different levels, thereby continuously improving the execution accuracy of the present invention, thereby reducing the complexity of data processing while taking into account accuracy and real-time performance.

[0109] In practical applications, with the continuous updating of the responsibility frame database and the interference frame database, the number of responsibility frames and interference frames of gas shielded welding will become larger and larger. When the number reaches a certain preset value, the first frame similarity and the second frame similarity mentioned in the above embodiment will become factors restricting the efficiency of the algorithm. Because after the number increases sharply, the number of times the first frame similarity and the second frame similarity are calculated will also increase sharply.

[0110] In order to eliminate this "data saturation" effect, this embodiment further considers that the values ​​of the responsible frames and interference frames stored in the responsible frame database and the interference frame database are actually different.

[0111] Specifically, the value of the responsibility frame in the responsibility frame database is proportional to the time, that is, the later the responsibility frame is saved in the responsibility frame database, the higher its value, because it is closer to the latest living habits of the currently monitored lung cancer patients; the value of the interference frame in the interference frame database is inversely proportional to the time, that is, the later the responsibility frame is saved in the responsibility frame database, the lower its value, because the interference is less.

[0112] Based on this, the further improvement method of this embodiment includes:

[0113] When the number of responsible frames A contained in the responsible frame database is greater than the preset value M, the responsible frames of number A are divided into hash groups and non-hash groups according to the frame storage time; A>M≥10 4 ;

[0114] The storage time of A1 responsible frames included in the hash group is earlier than the storage time of A2 responsible frames included in the non-hash group. Indicates rounding up;

[0115] For A1 responsible frames included in the hash group, generate corresponding A1 hash code values;

[0116] When the number of interference frames B contained in the interference frame database is greater than a preset value M, the number of interference frames B is divided into hash groups and non-hash groups according to the frame storage time; B>M≥10 4 ;

[0117] The storage time of B1 responsible frames included in the hash group is earlier than the storage time of B2 responsible frames included in the non-hash group. Indicates rounding up;

[0118] For the A1 responsible frames included in the hash group, generate corresponding A1 hash code values.

[0119] The step S120 further includes:

[0120] At this time, in a scenario, if the frame difference between two consecutive video frames Fa and Fb in the consecutive video frames is greater than a preset frame difference threshold, the hash code values ​​HFa and HFb corresponding to the video frames Fa and Fb are calculated;

[0121] If the hash code value HFa or HFb is equal to any one of the A1 hash code values, both the video frame Fa and the video frame Fb are used as the pain monitoring target frames.

[0122] In another scenario, if the frame difference between two consecutive video frames Fa and Fb in the consecutive video frames is greater than a preset frame difference threshold, the hash code values ​​HFa and HFb corresponding to the video frames Fa and Fb are calculated;

[0123] If the hash code value HFa or HFb is equal to any one of the B1 hash code values, the video frame Fa and the video frame Fb are not used as the pain monitoring target frame.

[0124] It can be seen that when the number of responsible frames and interference frames reaches a certain preset value, this embodiment no longer calculates multiple similarities one by one, omits this similarity calculation process, and adopts the hash code value comparison method. It can be predicted that the simple hash value comparison process is very fast and is basically not affected by the number of values.

[0125] Correspondingly, the hash value calculation process can also be advanced to the AI ​​monitoring engine stage, that is, step S140 is further improved as follows:

[0126] S140: The AI ​​monitoring engine analyzes the pain monitoring target frame to determine whether the lung cancer patient experiences pain;

[0127] When it is determined that the lung cancer patient has pain, if the number of responsible frames A contained in the responsible frame database is greater than a preset value M, the pain monitoring target frame is marked as a responsible frame, and a hash code value of the responsible frame is generated and saved in the responsible frame database;

[0128] When it is determined that the lung cancer patient has pain, if the number of interference frames contained in the interference frame database is greater than the preset value M, the pain monitoring target frame is marked as an interference frame, and a hash code value of the interference frame is generated and saved in the interference frame database.

[0129] It can be understood that generating a corresponding hash code value for each picture or video frame belongs to the prior art in the field.

[0130] Therefore, the above-mentioned further improved embodiment can further avoid the algorithm complexity caused by the automatic update of the image corpus of the responsible frame database and the interference frame database, thereby reducing the complexity of data processing while taking into account accuracy and real-time performance.

[0131] The aforementioned cloud-based pain monitoring method for lung cancer patients can be automatically implemented through various forms of electronic devices through computer-readable program instructions; the computer-readable program instructions can be stored in different forms of storage media and loaded into computer electronic devices for execution.

[0132] According to an exemplary implementation of the present disclosure, a computer-readable storage medium is provided, on which computer-executable instructions are stored, wherein the computer-executable instructions are executed by a processor to implement the method described above. According to an exemplary implementation of the present disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, and the computer-executable instructions are executed by a processor to implement the pain monitoring method for lung cancer patients described above.

[0133] correspond Figure 1 The method embodiment described, Figure 4 A schematic diagram of the functional modules of a pain monitoring system for lung cancer patients according to an embodiment of the present invention is further shown.

[0134] Figure 4 A pain monitoring system for lung cancer patients is shown, the system comprising an AI monitoring engine, a video frame acquisition unit, a pain monitoring target frame filtering unit, a responsibility frame database and an interference frame database.

[0135] The system further comprises:

[0136] A video frame acquisition unit, configured to acquire a plurality of continuous video frames including the posture of the lung cancer patient according to a preset period; the posture of the lung cancer patient includes the posture of limbs; and the posture of limbs includes the movements of limbs;

[0137] a pain monitoring target frame filtering unit, configured to determine a pain monitoring target frame in the continuous video frames based on a responsible frame, an interference frame, and an inter-frame difference between a plurality of continuous video frames, and send the pain monitoring target frame to the AI ​​monitoring engine;

[0138] The inter-frame difference is the inter-frame difference of the body posture of two consecutive video frames;

[0139] The AI ​​monitoring engine analyzes the pain monitoring target frame to determine whether the lung cancer patient experiences pain;

[0140] When it is determined that the lung cancer patient has pain, marking the pain monitoring target frame as a responsibility frame and saving it to a responsibility frame database;

[0141] Otherwise, the pain monitoring target frame is marked as an interference frame and saved in the interference frame database.

[0142] The system may also include a hash coding unit for generating a corresponding hash coding value for the marked pain monitoring target frame when relevant conditions are met (ie, the number of interference frames contained in the interference frame database or the number of responsible frames A contained in the responsible frame database is greater than a preset value M).

[0143] The lung cancer patient's posture also includes a facial posture; the facial posture includes a facial expression posture;

[0144] The AI ​​monitoring engine analyzes the pain monitoring target frame to determine whether the lung cancer patient experiences pain, specifically including:

[0145] The AI ​​monitoring engine determines whether the lung cancer patient is experiencing pain based on the correlation between the facial expression contained in the pain monitoring target frame and the pain level.

[0146] The pain monitoring target frame filtering unit determines the pain monitoring target frame in the continuous video frames based on the responsible frame, the interference frame and the frame difference between the multiple continuous video frames, specifically including:

[0147] If the frame difference between two consecutive video frames Fa and Fb in the consecutive video frames is less than a preset frame difference threshold, then the video frames Fa and Fb are not used as the pain monitoring target frames;

[0148] In one scenario, if the inter-frame difference between two consecutive video frames Fa and Fb in the consecutive video frames is greater than a preset inter-frame difference threshold, then determining the first inter-frame similarity between the video frame Fa or the video frame Fb and any of the responsible frames;

[0149] When the similarity between any of the first frames is greater than a first preset similarity threshold, both the video frame Fa and the video frame Fb are used as the pain monitoring target frames;

[0150] In another scenario, if the inter-frame difference between two consecutive video frames Fa and Fb in the consecutive video frames is greater than a preset inter-frame difference threshold, then determining the second inter-frame similarity between the video frame Fa or the video frame Fb and any of the interference frames;

[0151] When any of the second inter-frame similarities is greater than a second preset similarity threshold, the video frame Fa and the video frame Fb are not used as the pain monitoring target frames.

[0152] Based on the above improvements of this embodiment, the technical solution of the present application divides the pain monitoring process into a limb frame difference filtering stage and an expression recognition confirmation stage when performing pain recognition.

[0153] The above-mentioned limb frame difference filtering stage can filter out the responsible frames that may contain pain symptoms based on the multiple video frames originally captured to enter the subsequent expression recognition confirmation stage. Compared with the defects of the prior art that frame-by-frame recognition leads to a sharp increase in data processing volume while achieving half the result with twice the effort, this embodiment can significantly reduce the data processing volume (only Figure 3 The example shown has already reduced the amount of data by 20%. Considering other scenarios and the increase in the base sample size, the reduction in data volume will be more significant).

[0154] At the same time, the above process is based on the calculation method of the frame difference of limb posture, which can significantly reduce the amount of graphics data processing. Because when the limb posture changes, the anchor calculation area changes significantly, and there is no need for high-resolution contour recognition and pixel segmentation.

[0155] When the present embodiment is actually implemented, a basic responsible frame database and an interference frame database may be pre-established. As the present method is gradually executed, the responsible frame database and the interference frame database implement self-learning feedback updates, namely the above-mentioned marking process.

[0156] According to the above-mentioned marking process, when applying the relevant AI monitoring engine, the responsible frame database and the interference frame database can be further updated based on the results of the AI ​​monitoring engine, thereby realizing the full-process closed-loop feedback self-learning update process of the method, so that the image corpus of the responsible frame database and the interference frame database is automatically enriched at different levels, thereby continuously improving the execution accuracy of the present invention, thereby reducing the complexity of data processing while taking into account accuracy and real-time performance.

[0157] For other technologies, principles, algorithms or models not elaborated in detail in this application, please refer to the prior art.

[0158] In the above-mentioned embodiment section, the present invention provides multiple embodiments, each of which can constitute an independent technical solution and may contribute to the prior art and solve corresponding technical problems. However, it should be pointed out that different embodiments can be combined with each other without violating logic; at the same time, each embodiment can solve at least one technical problem, but it is not required that each individual embodiment solves multiple or all technical problems.

[0159] At the same time, in the specific implementation of this application, if user-related data is involved, when the embodiment of this application is applied to a specific product or technology, the user's permission or consent must be obtained, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0160] The above descriptions of various implementations of the present disclosure are exemplary, non-exhaustive, and not limited to the disclosed implementations. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described implementations. The selection of terms used herein is intended to best explain the principles of the implementations, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the various implementations disclosed herein.

Claims

1. A pain monitoring method for lung cancer patients, the method is implemented based on an edge image acquisition device, the image acquisition device is connected to an AI monitoring engine, and is characterized in that: The method comprises the following steps: S110: Acquire a plurality of continuous video frames including the posture of the lung cancer patient according to a preset period; S120: Determine a pain monitoring target frame in the continuous video frames; S130: Sending the pain monitoring target frame to the AI ​​monitoring engine; S140: The AI ​​monitoring engine analyzes the pain monitoring target frame to determine whether the lung cancer patient experiences pain; When it is determined that the lung cancer patient has pain, marking the pain monitoring target frame as a responsibility frame and saving it in a responsibility frame database; Otherwise, marking the pain monitoring target frame as an interference frame and saving it to the interference frame database; Wherein, the step S120 includes: Based on the responsible frame, the interference frame and the frame difference between a plurality of consecutive video frames, a pain monitoring target frame in the consecutive video frames is determined.

2. A pain monitoring method for lung cancer patients as claimed in claim 1, characterized in that: The lung cancer patient's posture in step S110 includes facial posture and limb posture; the facial posture includes facial expression posture, and the limb posture includes limb movements.

3. A pain monitoring method for lung cancer patients as claimed in claim 2, characterized in that: The step S120 further includes: If the frame difference between two consecutive video frames Fa and Fb in the consecutive video frames is less than a preset frame difference threshold, the video frames Fa and Fb are not used as the pain monitoring target frames.

4. A pain monitoring method for lung cancer patients as claimed in claim 2, characterized in that: The step S120 further includes: If the inter-frame difference between two consecutive video frames Fa and Fb in the consecutive video frames is greater than a preset inter-frame difference threshold, then determining the first inter-frame similarity between the video frame Fa or the video frame Fb and any of the responsible frames; When the similarity between any of the first frames is greater than a first preset similarity threshold, both the video frame Fa and the video frame Fb are used as the pain monitoring target frames.

5. A pain monitoring method for lung cancer patients as claimed in claim 2, characterized in that: The step S120 further includes: If the inter-frame difference between two consecutive video frames Fa and Fb in the consecutive video frames is greater than a preset inter-frame difference threshold, determining a second inter-frame similarity between the video frame Fa or the video frame Fb and any of the interference frames; When the similarity between any of the second frames is greater than a second preset similarity threshold, the video frame Fa and the video frame Fb are not used as the pain monitoring target frames.

6. A pain monitoring method for lung cancer patients according to any one of claims 3 to 5, wherein the inter-frame difference between the two consecutive video frames Fa and Fb is an inter-frame difference of limb postures included in the video frames Fa and Fb; The AI ​​monitoring engine analyzes the pain monitoring target frame to determine whether the lung cancer patient experiences pain, specifically including: The AI ​​monitoring engine determines whether the lung cancer patient is experiencing pain based on the correlation between the facial expression contained in the pain monitoring target frame and the pain level.

7. A pain monitoring method for lung cancer patients as claimed in claim 1, characterized in that: When the number A of responsible frames included in the responsible frame database is greater than a preset value M, the responsible frames of number A are divided into hash groups and non-hash groups according to the frame storage time; The storage time of A1 responsible frames included in the hash group is later than the storage time of A2 responsible frames included in the non-hash group; For A1 responsible frames included in the hash group, generate corresponding A1 hash code values; The step S120 further includes: If the frame difference between two consecutive video frames Fa and Fb in the consecutive video frames is greater than a preset frame difference threshold, then the hash code values ​​HFa and HFb corresponding to the video frames Fa and Fb are calculated; If the hash code value HFa or HFb is equal to any one of the A1 hash code values, both the video frame Fa and the video frame Fb are used as the pain monitoring target frames.

8. A pain monitoring method for lung cancer patients as claimed in claim 1, characterized in that: When the number of interference frames B included in the interference frame database is greater than a preset value M, the number B of interference frames are divided into hash groups and non-hash groups according to the frame storage time; The storage time of B1 interference frames included in the hash group is earlier than the storage time of B2 interference frames included in the non-hash group; For B1 interference frames included in the hash group, generate corresponding B1 hash code values; The step S120 further includes: If the frame difference between two consecutive video frames Fa and Fb in the consecutive video frames is greater than a preset frame difference threshold, then the hash code values ​​HFa and HFb corresponding to the video frames Fa and Fb are calculated; If the hash code value HFa or HFb is equal to any one of the B1 hash code values, the video frame Fa and the video frame Fb are not used as the pain monitoring target frame.

9. A pain monitoring system for lung cancer patients, the system comprising an AI monitoring engine, characterized in that: The system further comprises: A video frame acquisition unit, configured to acquire a plurality of continuous video frames including the posture of the lung cancer patient according to a preset period; the posture of the lung cancer patient includes the posture of limbs; and the posture of limbs includes the movements of limbs; a pain monitoring target frame filtering unit, configured to determine a pain monitoring target frame in the continuous video frames based on a responsible frame, an interference frame, and an inter-frame difference between a plurality of continuous video frames, and send the pain monitoring target frame to the AI ​​monitoring engine; The inter-frame difference is the inter-frame difference of the body posture of two consecutive video frames; The AI ​​monitoring engine analyzes the pain monitoring target frame to determine whether the lung cancer patient experiences pain; When it is determined that the lung cancer patient has pain, marking the pain monitoring target frame as a responsible frame; Otherwise, the pain monitoring target frame is marked as an interference frame.

10. A computer program product, comprising a computer program or a computer executable instruction, wherein when the computer program or the computer executable instruction is executed by a processor, the method for pain monitoring for lung cancer patients according to any one of claims 1 to 8 is implemented.

Images