Pain detection method, device and equipment based on infrared imaging and medium

Abstract

The invention relates to the technical field of image processing, in particular to a pain detection method, device and equipment based on infrared imaging and a medium. The method comprises the following steps: firstly, acquiring an original infrared image of an individual, preprocessing the original infrared image, then marking semantics for key parts on the infrared image, segmenting the key parts into key parts of pain, and analyzing a final pain detection result according to the infrared image of the pain parts and the body surface temperature. Since each key part is marked with semantics, namely, the position of the key part on the infrared image is positioned, and the corresponding relation between each key part on the infrared image and the body surface temperature can be established through the semantics, the position of pain can be positioned through the semantics, and the position of the pain can be determined by combining the body surface temperature. Therefore, key pain parts can be positioned, and the pain detection result can be improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a pain detection method, device, equipment and medium based on infrared imaging. Background Art

[0002] Images are obtained through X-ray, B-ultrasound, CT and other technologies, and pain can be analyzed by analyzing the images. Among them, the above technologies can only detect pain after the pain is formed. Infrared imaging technology can predict and measure pain using the following principles: before the disease causes changes in tissue structure and morphology, cell metabolism will be abnormal, and the human body will change in temperature. The temperature, the shape of the temperature field, and the size of the temperature difference can reflect the location, nature and degree of the disease. When a part of the human body is damaged, the infrared thermal imager can intuitively give a human body temperature distribution map. With the help of damaged tissue, the blood supply and oxygen consumption balance of the original tissue will be broken, causing a large change in the symmetry and consistency of the body temperature distribution, and achieving the measurement and prediction of pain and its location. However, since the existing technology does not pay attention to the semantic information of the key parts of the human body on the infrared image, the existing technology cannot locate the key parts of pain.

[0003] In summary, the existing technology cannot locate the key pain site.

[0004] Therefore, the prior art still needs to be improved and enhanced. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides a pain detection method, device, equipment and medium based on infrared imaging, which solves the problem that the prior art cannot locate the key pain site.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] In a first aspect, the present invention provides a pain detection method based on infrared imaging, which comprises:

[0008] Collecting an original infrared image of an individual, and preprocessing the original infrared image to obtain a preprocessed infrared image;

[0009] Based on the semantics of the key parts of the individual, marking the key parts on the preprocessed infrared image to obtain a semantically marked infrared image;

[0010] Segmenting the pain image corresponding to the key pain site on the semantically annotated infrared image;

[0011] The body surface temperature of the individual at the key pain site is obtained, and a pain detection result of the individual is obtained based on the pain image and the body surface temperature.

[0012] In one implementation, preprocessing the original infrared image to obtain a preprocessed infrared image includes:

[0013] Acquire environmental data of the individual, wherein the environmental data is environmental data that affects the quality of the infrared image; acquire evaluation indicators of the infrared image; acquire acquisition characteristics of the original infrared image, and acquire the type of noise caused by the acquisition characteristics;

[0014] The original infrared image is preprocessed according to the environmental data, the evaluation index, and the noise type to obtain a preprocessed infrared image.

[0015] In one implementation, the step of labeling the key parts on the preprocessed infrared image based on the key parts semantics of the individual to obtain the semantically labeled infrared image includes:

[0016] Obtain the semantic benchmark annotation system of key parts and the dimension information of key parts;

[0017] According to the reference marking system and the size information, traverse the pre-processed infrared image to mark key parts on the pre-processed infrared image;

[0018] Acquiring the motion information of the individual when the original infrared image is collected; and / or acquiring the motion information of the device that collects the original infrared image;

[0019] According to the motion information of the individual and / or the motion information of the acquisition device, the pre-processed infrared image is subjected to motion blur removal processing, and the pre-processed infrared image after key parts are marked and motion blur is removed is used as a semantically annotated infrared image.

[0020] In one implementation, segmenting the pain image corresponding to the key pain site on the semantically annotated infrared image includes:

[0021] Acquiring a baseline infrared image of a key part of the individual;

[0022] Pre-segmenting the semantically annotated infrared image according to the reference infrared image;

[0023] A deep learning network and an instance segmentation network are sequentially applied to the semantically annotated infrared image after pre-segmentation to obtain a pain image.

[0024] In one implementation, the deep learning network is a trained deep learning network, and constructing a reward function required for training the deep learning network includes:

[0025] Obtain a downsampled infrared image of the sample infrared image, obtain a first key part segmentation result of the sample infrared image, and obtain a second key part segmentation result of the downsampled infrared image; the first key part segmentation result is the segmentation result output by the deep learning network when the sample infrared image is input into the deep learning network; the second key part segmentation result is the segmentation result output by the deep learning network when the downsampled infrared image is input into the deep learning network;

[0026] A true segmentation result of the key parts on the sample infrared image is obtained, and a reward function is constructed according to the first key part segmentation result, the second key part segmentation result and the true segmentation result.

[0027] In one implementation, obtaining the body surface temperature of the individual at the key pain site, and obtaining the pain detection result of the individual according to the pain image and the body surface temperature, includes:

[0028] Acquiring a body surface reference temperature, and constructing three-dimensional reconstruction data corresponding to the key pain site based on the body surface reference temperature, the pain image, and the body surface temperature;

[0029] Pre-screening the key pain sites according to the three-dimensional reconstruction data to obtain the key pain pre-screening sites;

[0030] Obtaining the position of the pain pre-screening key part on the individual, and generating a key part sub-graph corresponding to the pain pre-screening key part according to the position;

[0031] According to the key part sub-graph, the pain pre-screening key parts are screened to obtain the pain preliminary selection key parts;

[0032] Applying a Bayesian decision algorithm to the three-dimensional reconstruction data of the preliminary selected key pain sites to screen out the target key pain sites from the preliminary selected key pain sites;

[0033] The pain degree of the key parts of the pain target is quantitatively analyzed to obtain the pain detection result of the individual.

[0034] In one implementation, the method further includes:

[0035] Obtaining influencing factors that cause pain, and establishing a pain causal inference model based on the influencing factors;

[0036] Obtaining the time corresponding to the pain detection result of the individual, and predicting the pain development trend of the individual based on the time;

[0037] Evaluation indicators for the pain development trend are established through graph neural networks.

[0038] In a second aspect, an embodiment of the present invention further provides a pain detection device based on infrared imaging, wherein the device comprises the following components:

[0039] A preprocessing module, used for collecting original infrared images of individuals and preprocessing the original infrared images to obtain preprocessed infrared images;

[0040] A labeling module, used for labeling the key parts on the preprocessed infrared image based on the semantics of the key parts of the individual, to obtain a semantically labeled infrared image;

[0041] A segmentation module, used for segmenting the pain image corresponding to the key pain site on the semantically annotated infrared image;

[0042] The detection module is used to obtain the body surface temperature of the individual at the key pain site, and obtain the pain detection result of the individual based on the pain image and the body surface temperature.

[0043] In a third aspect, an embodiment of the present invention further provides a terminal device, wherein the terminal device includes a memory, a processor, and an infrared imaging-based pain detection program stored in the memory and executable on the processor, and when the processor executes the infrared imaging-based pain detection program, the steps of the above-mentioned infrared imaging-based pain detection method are implemented.

[0044] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which is stored an infrared imaging-based pain detection program. When the infrared imaging-based pain detection program is executed by a processor, the steps of the above-mentioned infrared imaging-based pain detection method are implemented.

[0045] Beneficial effects: The present invention first collects the original infrared image of the individual, and pre-processes the original infrared image, then annotates the key parts on the infrared image with semantics, and divides the key parts into key parts of pain, and analyzes the final pain detection results according to the infrared image and body surface temperature of the painful part. Since the present invention annotates the semantics for each key part, that is, locates the position of the key part on the infrared image, and can establish a corresponding relationship between each key part on the infrared image and the body surface temperature through the semantics, the location of the pain can be located through the semantics, and combined with the body surface temperature, the present invention can both locate the key parts of pain and improve the pain detection results. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is the overall flow chart of the present invention;

[0047] Figure 2A schematic diagram of key parts in an embodiment of the present invention;

[0048] Figure 3 is a flowchart of infrared imaging information extraction in an embodiment of the present invention;

[0049] Figure 4 This is a flowchart of quantitatively assessing pain status in an embodiment of the present invention;

[0050] Figure 5 This is a flow chart of post-healing pain development trend assessment in an embodiment of the present invention;

[0051] Figure 6 This is a structural diagram of the pain detection device based on infrared imaging provided by the present invention;

[0052] Figure 7 This is a block diagram of the internal structure principle of the terminal device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0053] The following is a clear and complete description of the technical solution of the present invention in combination with the embodiments and the accompanying drawings. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0054] Research has found that pain can be analyzed by obtaining images through X-ray, B-ultrasound, CT and other technologies, and analyzing the images. Among them, the above technologies can only detect pain after the pain is formed. Infrared imaging technology can predict and measure pain using the following principles: before the disease causes changes in tissue structure and morphology, cell metabolism will be abnormal, and the human body will change in temperature. The temperature, the shape of the temperature field, and the size of the temperature difference can reflect the location, nature and degree of the disease. When a part of the human body is damaged, the infrared thermal imager can intuitively give a human body temperature distribution map. With the help of damaged tissue, the blood supply and oxygen consumption balance of the original tissue will be broken, causing a large change in the symmetry and consistency of the body temperature distribution, and achieving the measurement and prediction of pain and its location. However, since the existing technology does not pay attention to the semantic information of the key parts of the human body on the infrared image, the existing technology cannot locate the key parts of pain.

[0055] In order to solve the above technical problems, the present invention provides a pain detection method, device, equipment and medium based on infrared imaging, which solves the problem that the prior art cannot locate the key pain parts. In specific implementation, firstly, the original infrared image of the individual is collected, and the original infrared image is preprocessed to obtain a preprocessed infrared image; then, based on the semantics of the key parts of the individual, the key parts on the preprocessed infrared image are annotated to obtain a semantically annotated infrared image; then, the pain image corresponding to the key pain parts on the semantically annotated infrared image is segmented; finally, the body surface temperature of the individual at the key pain parts is obtained, and the pain detection result of the individual is obtained based on the pain image and the body surface temperature.

[0056] For example, taking the human body as an example, the original infrared image of the human cervical spine is collected, and the original infrared image is preprocessed to obtain a preprocessed infrared image, and then the key parts of the preprocessed infrared image are annotated, that is, the semantics of the key parts are annotated, such as the key parts include Figure 2 The C3 transverse process (located at the junction of the horizontal line of the hyoid angle and the posterior edge of the sternocleidomastoid muscle), C4 transverse process (located at the junction of the horizontal line of the proximal upper edge of the thyroid cartilage and the posterior edge of the same muscle), C5 transverse process (located at the junction of the horizontal line of the thyroid cartilage and the posterior edge of the same muscle) and C6 transverse process (located at the junction of the horizontal line of the cricoid cartilage and the posterior edge of the same muscle) are shown. According to the infrared images of the above four key parts, analyze which key parts belong to the pain parts. If the result of the analysis is that the C3 transverse process and the C4 transverse process are the pain parts, then segment the image area (pain image) where the C3 transverse process is located and the image area (pain image) where the C4 transverse process is located from the semantically annotated infrared image. Collect the body surface temperatures of the two parts of the C3 transverse process and the C4 transverse process, and analyze the pain detection results of the C3 transverse process according to the pain image corresponding to the C3 transverse process and the body surface temperature of the C3 transverse process, and analyze the pain detection results of the C4 transverse process according to the pain image corresponding to the C4 transverse process and the body surface temperature of the C4 transverse process.

[0057] The infrared imaging-based pain detection method of this embodiment can be applied to a terminal device, which can be a terminal product with an image processing function, such as a pain detector. Figure 1 As shown in , the pain detection method based on infrared imaging specifically includes the following steps:

[0058] S100, collecting an original infrared image of an individual, and preprocessing the original infrared image to obtain a preprocessed infrared image;

[0059] S200, based on the semantics of the key parts of the individual, annotating the key parts on the pre-processed infrared image to obtain a semantically annotated infrared image;

[0060] S300, segmenting the pain image corresponding to the key pain site on the semantically annotated infrared image;

[0061] S400, obtaining the body surface temperature of the individual at the key pain site, and obtaining a pain detection result of the individual based on the pain image and the body surface temperature.

[0062] Embodiment 1. In this embodiment, step S100 includes the following specific steps: obtaining environmental data of the individual, wherein the environmental data is environmental data that affects the quality of the infrared image; obtaining evaluation indicators of the infrared image; obtaining acquisition characteristics of the original infrared image, and obtaining the type of noise caused by the acquisition characteristics; preprocessing the original infrared image according to the environmental data, the evaluation indicators, and the noise type to obtain a preprocessed infrared image.

[0063] like Figure 3 As shown, the present application can collect original infrared images of the human head, upper limbs, chest, abdomen and lower limbs. Any original infrared image needs to be preprocessed to improve the quality of the infrared image to facilitate subsequent pain detection.

[0064] Among them, the environmental data includes temperature and humidity. The infrared image is filtered by temperature and humidity to remove the noise interference caused by temperature and humidity to the infrared image.

[0065] Different acquisition characteristics (acquisition characteristics can be the type of acquisition equipment) will bring different types of noise to the image. Therefore, the noise type can be analyzed through the acquisition characteristics. After knowing the noise type of the image, the corresponding denoising algorithm can be selected to remove the noise on the image. For example, signal reconstruction and resampling can be used to unify the distribution of infrared information collected by signals at different times. Through scale and dimensional transformation, the expression forms of multi-source signals can be unified to overcome the influence of various external noises on the signals.

[0066] The evaluation index is constructed in the following ways: on the basis of unified multi-time multi-source signals, the random factors such as measurement errors and noise interference accompanying the monitoring process are analyzed, and on the basis of the consistency of temperature distribution of the same object in a short period of time, the quality evaluation indexes of the accuracy and integrity of the data information are established. That is, multiple quality evaluation indexes such as accuracy and integrity are established, infrared images are evaluated according to the quality evaluation indexes, and infrared images that can be used for subsequent pain detection are screened according to the evaluation results.

[0067] This embodiment also selects high-quality data through clustering, classification, etc., removes redundant and low-quality data, and improves the quality of infrared thermal image data.

[0068] Embodiment 2, based on embodiment 1, in this embodiment, S200 includes the following specific steps S201, S202, S203, S204:

[0069] S201, obtaining a semantic reference annotation system of key parts and dimension information of key parts.

[0070] The key part in this embodiment is Figure 2 The key parts annotated in the embodiment are the names of each key part, and the key part semantics is the name of each key part. That is, this embodiment combines the clinical semantic information of the key parts of the human body to construct a clinical body key part annotation system, and uses clinical prior knowledge as a guide to construct the key part granularity information (that is, size information).

[0071] S202: traverse the preprocessed infrared image according to the reference marking system and the size information to mark key parts on the preprocessed infrared image.

[0072] Slide the window from left to right and from top to bottom, cut out fixed-size infrared thermal image blocks based on the clinical semantic information of key parts of the body, and annotate the temperature value on each image. That is, one semantic corresponds to one infrared thermal image block, and annotate the infrared temperature and semantics for each infrared thermal image block.

[0073] S203, obtaining the motion information of the individual when the original infrared image is collected; and / or obtaining the motion information of the device for collecting the original infrared image.

[0074] S204, performing motion blur removal processing on the pre-processed infrared image according to the motion information of the individual and / or the motion information of the acquisition device, and using the pre-processed infrared image after key parts are marked and motion blur is removed as a semantically annotated infrared image.

[0075] The movement of the acquisition device and the movement of the individual will cause the infrared image to be mixed with noise data to form a blurred image, so this part of the noise data needs to be removed to improve the quality of the infrared image. In this embodiment, an adversarial network is used to deblur the infrared image, including the following specific steps: first, the clarity of the image is calculated based on the edge detection principle. The blurred image with a small edge gradient amplitude is filtered out by a threshold; secondly, a generative adversarial network is used to achieve a direct mapping between the blurred image and the clear image to remove the blur noise.

[0076] Among them, the generative adversarial network is a trained generative adversarial network. The loss functions used to train the generative adversarial network include content loss and adversarial loss. The former evaluates the gap between the restored image and the real clear image, and the latter trains the generator and discriminator in an alternating manner.

[0077] If the discriminator during training cannot distinguish between the restored image and the real clear image, it is considered that the performance of the generator has reached the best. Therefore, by training the generator and the discriminator, the training of the generative adversarial network is completed, so that the trained generative adversarial network can remove the motion noise on the infrared image to obtain a clear temperature distribution map of the key parts.

[0078] Embodiment 3, based on embodiment 1 or embodiment 2, in this embodiment, step S300 includes the following specific steps: obtaining a baseline infrared image of the key parts of the individual; pre-segmenting the semantically annotated infrared image based on the baseline infrared image; applying a deep learning network and an instance segmentation network in sequence to the semantically annotated infrared image after pre-segmentation to obtain a pain image.

[0079] By semantically annotating infrared images, the temperature of each key part can be obtained, that is, there is a corresponding relationship between the pixels and temperature of the infrared image. The pixel values ​​of the infrared image are converted into a density map, and the probability of abnormality at the location of each key part on the density map is calculated. The discrete measured temperature distribution is generated by morphological expansion and connected domain extraction. The infrared image of the key part without pain is obtained, and the infrared image is used as a benchmark, and the temperature distribution of the benchmark infrared image is calculated. The measured temperature distribution is compared with the benchmark temperature distribution to determine whether the measured temperature distribution is abnormal, and then whether the key part is painful based on the abnormal judgment result. If it is determined that there is a key pain part on the infrared image, the pain part is segmented. After segmentation, the key pain part on the infrared image is further segmented through a deep learning network (that is, a deep Q learning network) and an instance segmentation network to obtain the final segmented image (that is, a pain image).

[0080] In this embodiment, based on the temperature distribution labeling method of deep Q learning, the state s of the deep Q learning network is defined as an accuracy gain map, which represents the accuracy difference of temperature anomaly detection on high-resolution and low-resolution images respectively, and the action α is defined as extracting the high-definition thermal image block of the original thermal image.

[0081] The reward function for action α is R:

[0082]

[0083] In the formula, is the segmentation result of the dth key part on the sample infrared image (the first key part segmentation result), is the segmentation result of the dth key part on the downsampled infrared image (the second key part segmentation result), g dis the true segmentation result of the key part, b and B are the total number of pixels in the extracted image area and the original image respectively, and λ is a constant, which is used to control the trade-off between accuracy and computational complexity.

[0084] This embodiment can also use an instance segmentation model to segment key parts on an image. The instance segmentation model consists of a single-stage detection network and a mask prediction subnetwork. The single-stage network predicts a rectangular box for each key point, and the mask prediction subnetwork outputs a pixel segmentation map of the defect. The combination of the two can generate a high-quality mask for each key part object. The loss function of the instance segmentation model = confidence loss + classification loss + positioning loss + mask loss, where the sum of confidence loss, classification loss, and positioning loss is the loss function of the single-stage detection network, and the mask loss is the loss function of the mask prediction subnetwork, where the mask loss uses pixel-by-pixel cross entropy to calculate the loss.

[0085] This embodiment can optimize the model from the number of output scales of the single-stage detection network and the number of repetitive residual units in the backbone architecture to improve the accuracy and speed of the model; it can also optimize the model from the connection method of the single-stage network and the mask prediction subnetwork to improve the accuracy and speed of the model; it can also introduce the rotation angle prediction of the key part box to reduce the background redundant area in the prediction box, so as to more accurately locate the key parts of the human body, so as to improve the accuracy and speed of the model.

[0086] In the fourth embodiment, the above-mentioned infrared image is used to segment the possible painful parts, but since a single thermal distribution map is difficult to reflect the specific pain information of an individual, the individual's pain level cannot be obtained based on the thermal distribution map. Based on the first embodiment, the second embodiment or the third embodiment, this embodiment further analyzes the individual's pain level by combining the body surface temperature. In this embodiment, step S400 includes the following specific steps S301 to S306:

[0087] S301, obtaining a body surface reference temperature, and constructing three-dimensional reconstruction data corresponding to the key pain site based on the body surface reference temperature, the pain image, and the body surface temperature.

[0088] That is, the three-dimensional reconstruction data of the same key part includes the body surface baseline temperature (that is, the epidemic body surface temperature distribution), the pain image, and the individual's body surface temperature.

[0089] The body surface reference temperature is constructed as follows: Figure 4 As shown in the figure, the thermal image information of key parts of the body under a large number of pathological conditions directly uses a convolutional neural network to predict the Gaussian distribution probability obeyed by the key points of each key part of the body. Combined with the clinical key part marking system of the body, a body temperature distribution benchmark data set with different Gaussian image probability distributions is statistically obtained.

[0090] S302: Pre-screen the key pain sites according to the three-dimensional reconstruction data to obtain pre-screened key pain sites.

[0091] The three-dimensional reconstruction data includes the body surface baseline temperature (that is, the epidemic body surface temperature distribution), the pain image, and the individual's body surface temperature. If the body surface temperature of the key part corresponding to the pain image is different from the body surface baseline temperature, then the key part corresponding to the pain image is the pain pre-screening key part, otherwise the key part corresponding to the pain image is not the pain pre-screening key part.

[0092] S303, obtaining the position of the pain pre-screening key part on the individual, and generating a key part sub-graph corresponding to the pain pre-screening key part according to the position.

[0093] S304, screening the pain pre-screening key parts according to the key part sub-graph to obtain the pain preliminary key parts.

[0094] For example, five pain pre-screening key parts are screened out through steps S301 and S302. These five pain pre-screening key parts are located in the head, hands, feet, arms and legs of the human body, among which the hands and arms are connected, and the connection between the two constitutes a key part subgraph; the feet and legs are also connected, and the connection between the two also constitutes a key part subgraph. The key parts that can constitute a subgraph are related parts, which will affect each other. Therefore, the hands, feet, arms and legs are the key parts for the initial selection of pain. There is no correlation between the head and the hands, feet, arms and legs, so the head is not a key part for the initial selection of pain.

[0095] It is also possible to further determine whether pain may occur in the selected key pain areas based on the age and gender of the human body, that is, to further confirm the key pain areas using age and gender.

[0096] S305, applying a Bayesian decision algorithm to the three-dimensional reconstruction data of the preliminary selected key pain sites to screen out target pain key sites from the preliminary selected key pain sites.

[0097] That is, the Bayesian decision algorithm is used to calculate the probability of pain occurring in each key part, and based on the probability, it is determined whether the primary key part of pain is the actual part where pain occurs.

[0098] For example, if the pain key part appears on n infrared images, y is used to represent the pain key part, y={(i,Y i )i∈{1,...,n},Y i∈{NO,LC,HC}}, where NO means that the anomaly is not detected on a certain subgraph, LC (LowConfidence) is low confidence, and HC (HighConfidence) is high confidence.

[0099] In this embodiment, the detection results of different images are independent of each other, and the posterior probability ratio can be calculated to determine whether they belong to the real pain occurrence candidate set:

[0100]

[0101] Among them, AD, They represent real and non-real temperature anomalies respectively; is the prior probability ratio; P(Y i |AD), The real and non-real temperature anomalies are detected as Y i The probability, P(Y i |AD), It can be obtained from the testing phase of the instance segmentation model, and the prior probability ratio can be estimated through actual clinical epidemiological statistical surveys.

[0102] If the posterior probability ratio of the pain preliminary key part y is greater than 1, the pain preliminary key part y is the pain target key part, that is, the pain preliminary key part y is the actual pain site. This is because the closer the confidence value is to 1, the more correct the segmentation result is, and 1 is the threshold.

[0103] This embodiment can also identify the key pain site according to the left-right symmetry of the infrared image.

[0104] S306, quantitatively analyzing the pain degree of the key parts of the pain target to obtain the pain detection result of the individual.

[0105] After it is determined through steps S301 to S305 that pain has actually occurred, the same pain semantic abnormalities on multiple images can be merged to generate a complete pain state description model, and combined with the semantic information of the clinical three-dimensional body model disease, a detailed quantitative analysis of the pain state is performed. That is, through the semantic information of the clinical three-dimensional body model disease, the clinical manifestations of pain in different parts, such as pain sensation, onset time, type and other information, are obtained. For different types of pain in different parts of the body, a correlation between different types of pain in different parts and the temperature distribution of the body thermal image is established through a large number of epidemiological statistics, and a mapping model is established using a neural network or the like. According to the relative mapping relationship between the clinical disease semantic information and the body thermal image, the distribution model of key parts of the human body is used to associate the pain state data information (such as pain type, onset time, diagnosis and treatment symptoms, etc.) with the temperature distribution law, and a quantitative evaluation of the pain thermal image, information archiving and visual diagnosis results are established.

[0106] Embodiment 5, based on embodiment 1 or embodiment 2 or embodiment 3 or embodiment 4, as Figure 5 As shown, this embodiment establishes a pain causal inference model (i.e. Figure 5 The causal representation model of pain state in the pain state is used to analyze the causes of pain). The establishment of the pain causal inference model includes: combining clinical semantic information, taking the post-healing effect evaluation of the target pain state as the final node, analyzing the 5W1H attributes of the pain state, and constructing a causal representation model for inferring results from causes. The 5W1H attributes include the five attributes of cause Why, description What, association Who, time When, place Where, and result How. Among them, the cause Why, description What, association Who, time When, and place Where are the influencing shadows of pain.

[0107] This implementation example Figure 5 As shown, predicting the pain development trend of the individual includes: obtaining the time when the individual suffers from pain, the pain type and the individual's attribute information, and predicting the pain development trend of the individual.

[0108] The development trend of pain in an individual can also be predicted based on the difference between the infrared image before treatment and the infrared image after treatment. Alternatively, the development trend of pain in an individual can be predicted based on the difference between the infrared image before treatment, the infrared image during treatment, and the infrared image after treatment. The differences include Euclidean distance, mean square distance, Mahalanobis distance, correlation coefficient, bulldozer distance, etc.

[0109] Incremental algorithms can also be applied to infrared images of individuals before treatment, infrared images of individuals during treatment, infrared images of individuals after treatment, baseline infrared images (infrared images of individuals without pain), infrared images corresponding to each type of pain, and individual attributes to predict the development trend of pain in that individual.

[0110] This implementation example Figure 5 As shown, multi-time series causal analysis, Bayesian network, etc. are used to construct a "temperature distribution difference-pain state development" mapping association mechanism model to achieve modeling of the temperature difference correlation relationship between the occurrence and development of pain, providing a basis for the subsequent evaluation of the occurrence and development of pain.

[0111] This embodiment establishes an evaluation index for the development trend of pain, and uses the evaluation index to characterize the pain level that an individual may experience in the future. That is, the temperature difference between the individual's body surface temperature and the reference temperature is input into the trained graph neural network, and the result y output by the graph neural network can be used to characterize the pain level that the individual may experience in the future. Among them, each node of the graph neural network is each key part of the individual.

[0112]

[0113] x n is the temperature difference corresponding to the key part n (temperature difference before treatment), k is the current layer number; N(n) is the neighbor of node u; f(·) is the activation or update function, f(·) includes sigmoid and Relu; is the aggregate information of neighbor node u at layer k-1; is the new feature vector of node n at layer k (i.e., the temperature difference corresponding to the treatment); are the feature vector of node v and its corresponding i-layer parsing step to be learned, Θ, W are the parameters to be learned by aggregating neighbor node information, that is, Θ and W are parameters that graph neural networks need to train and learn.

[0114] The "large margin nearest neighbor" optimization learning mechanism can also be used to "increase the distance between classes and reduce the distance within classes" to construct multi-dimensional evaluation indicators for the pain state evaluation model.

[0115] Individual infrared thermal imaging Figure 1 Generally, information is collected before admission, during treatment, and after recovery. Since it is impossible to continuously collect the patient's temperature distribution in real time, there is a lack of continuous temperature distribution information. The quantitative evaluation of the efficacy of pain recovery is the core of assisting clinical rapid and efficient pain diagnosis and treatment. It is urgent to propose an analysis method for the efficacy of pain recovery and its development trend of the body with insufficient information. In other words, how to predict the pain state and development trend of an individual after treatment when there are only infrared images before treatment and infrared images during treatment, but no infrared images after treatment.

[0116] That is, by combining the clinical semantic information of the key parts of the body (that is, the infrared images of the key parts before treatment and the infrared images during treatment), the development trend of pain after treatment is predicted.

[0117] That is to analyze the changes in the development trend of pain, influencing factors, etc., through the behavior of correlation changes over time, with the help of temperature distribution difference-pain state development description, to form a dynamic evolution mechanism of pain recovery based on time series relationship.

[0118] On the basis of the temperature distribution difference-pain state evaluation system, and on the basis of the correlation between indicators such as clinical semantic consistency, evaluation reliability, causality and explainability, the commonalities and differences in the development of pain before, during and after treatment are analyzed, and the development trend behavioral trajectory of pain after recovery is constructed.

[0119] Based on the regularity and sensitivity of the temperature distribution difference-pain state mapping, combined with the clinical semantic information of key parts of the body, the spatiotemporal evolution trend of the pain occurrence and development process is visualized. The interpretability and accuracy of the evaluation process can be displayed from multiple angles, such as radar charts and sunburst charts, but not limited to them.

[0120] In summary, the pain location detection method constructed by the present invention focuses on the identification of key parts of the human body, combines the temperature distribution differences between normal human bodies and patients, and the approximate similarities between key parts of the human body, so as to evaluate the abnormal parts of the human body temperature distribution and give the possible parts of the human body where pain occurs. This method is based on the information of the human body thermal radiation map, and seamlessly connects the clinical diagnosis and analysis of the disease in a non-toxic and non-invasive way to make a correct and objective diagnosis.

[0121] The present invention is based on a method for fusing temperature distribution information in multiple thermal images based on Bayesian decision making. Based on the semantic information of diseases in a three-dimensional body model, the mapping relationship between temperature distribution information and the occurrence and development of pain is calculated, thereby further calculating the pain probability of candidate parts and providing a theoretical basis for efficient clinical treatment.

[0122] This embodiment also provides a pain detection device based on infrared imaging, such as Figure 6 As shown, the device comprises the following components:

[0123] The preprocessing module 01 is used to collect the original infrared image of the individual and preprocess the original infrared image to obtain a preprocessed infrared image;

[0124] A labeling module 02 is used to label the key parts on the pre-processed infrared image based on the semantics of the key parts of the individual to obtain a semantically labeled infrared image;

[0125] The segmentation module 03 is used to segment the pain image corresponding to the key pain part on the semantically annotated infrared image;

[0126] The detection module 04 is used to obtain the body surface temperature of the individual at the key pain site, and obtain the pain detection result of the individual according to the pain image and the body surface temperature.

[0127] Based on the above embodiments, the present invention further provides a terminal device, whose principle block diagram can be shown as follows: Figure 7 As shown. The terminal device includes a processor, a memory, a network interface, and a display screen connected via a system bus. The processor of the terminal device is used to provide computing and control capabilities. The memory of the terminal device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the terminal device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a pain detection method based on infrared imaging is implemented. The display screen of the terminal device can be a liquid crystal display screen or an electronic ink display screen.

[0128] Those skilled in the art will understand that Figure 7 The principle block diagram shown in the figure is only a block diagram of a partial structure related to the scheme of the present invention, and does not constitute a limitation on the terminal device to which the scheme of the present invention is applied. The specific terminal device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0129] In one embodiment, a terminal device is provided. The terminal device includes a memory, a processor, and a pain detection program based on infrared imaging stored in the memory and executable on the processor. When the processor executes the pain detection program based on infrared imaging, the following operation instructions are implemented:

[0130] Collecting an original infrared image of an individual, and preprocessing the original infrared image to obtain a preprocessed infrared image;

[0131] Based on the semantics of the key parts of the individual, marking the key parts on the preprocessed infrared image to obtain a semantically marked infrared image;

[0132] Segmenting the pain image corresponding to the key pain site on the semantically annotated infrared image;

[0133] The body surface temperature of the individual at the key pain site is obtained, and a pain detection result of the individual is obtained based on the pain image and the body surface temperature.

[0134] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0135] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A pain detection method based on infrared imaging, characterized in that: include: Collecting an original infrared image of an individual, and preprocessing the original infrared image to obtain a preprocessed infrared image; Based on the semantics of the key parts of the individual, marking the key parts on the preprocessed infrared image to obtain a semantically marked infrared image; Segmenting the pain image corresponding to the key pain site on the semantically annotated infrared image; The body surface temperature of the individual at the key pain site is obtained, and a pain detection result of the individual is obtained based on the pain image and the body surface temperature.

2. The pain detection method based on infrared imaging as claimed in claim 1, characterized in that: The preprocessing of the original infrared image to obtain a preprocessed infrared image includes: Acquire environmental data of the individual, wherein the environmental data is environmental data that affects the quality of the infrared image; acquire evaluation indicators of the infrared image; acquire acquisition characteristics of the original infrared image, and acquire the type of noise caused by the acquisition characteristics; The original infrared image is preprocessed according to the environmental data, the evaluation index, and the noise type to obtain a preprocessed infrared image.

3. The pain detection method based on infrared imaging as claimed in claim 1, characterized in that: The step of labeling the key parts on the preprocessed infrared image based on the key parts semantics of the individual to obtain the semantically labeled infrared image includes: Obtain the semantic benchmark annotation system of key parts and the dimension information of key parts; According to the reference marking system and the size information, traverse the pre-processed infrared image to mark key parts on the pre-processed infrared image; Acquiring the motion information of the individual when the original infrared image is collected; and / or acquiring the motion information of the device that collects the original infrared image; According to the motion information of the individual and / or the motion information of the acquisition device, the pre-processed infrared image is subjected to motion blur removal processing, and the pre-processed infrared image after key parts are marked and motion blur is removed is used as a semantically annotated infrared image.

4. The pain detection method based on infrared imaging as claimed in claim 1, characterized in that: The segmenting of the pain image corresponding to the key pain site on the semantically annotated infrared image includes: Acquiring a baseline infrared image of a key part of the individual; Pre-segmenting the semantically annotated infrared image according to the reference infrared image; A deep learning network and an instance segmentation network are sequentially applied to the semantically annotated infrared image after pre-segmentation to obtain a pain image.

5. The pain detection method based on infrared imaging as claimed in claim 4, characterized in that: The deep learning network is a trained deep learning network, and the reward function required for training the deep learning network is constructed, including: Obtain a downsampled infrared image of the sample infrared image, obtain a first key part segmentation result of the sample infrared image, and obtain a second key part segmentation result of the downsampled infrared image; the first key part segmentation result is the segmentation result output by the deep learning network when the sample infrared image is input into the deep learning network; the second key part segmentation result is the segmentation result output by the deep learning network when the downsampled infrared image is input into the deep learning network; A true segmentation result of the key parts on the sample infrared image is obtained, and a reward function is constructed according to the first key part segmentation result, the second key part segmentation result and the true segmentation result.

6. The pain detection method based on infrared imaging as claimed in claim 1, characterized in that: Acquiring the body surface temperature of the individual at the key pain site, and obtaining the pain detection result of the individual according to the pain image and the body surface temperature, including: Acquiring a body surface reference temperature, and constructing three-dimensional reconstruction data corresponding to the key pain site based on the body surface reference temperature, the pain image, and the body surface temperature; Pre-screening the key pain sites according to the three-dimensional reconstruction data to obtain the key pain pre-screening sites; Obtaining the position of the pain pre-screening key part on the individual, and generating a key part sub-graph corresponding to the pain pre-screening key part according to the position; According to the key part sub-graph, the pain pre-screening key parts are screened to obtain the pain preliminary selection key parts; Applying a Bayesian decision algorithm to the three-dimensional reconstruction data of the preliminary selected key pain sites to screen out the target key pain sites from the preliminary selected key pain sites; The pain degree of the key parts of the pain target is quantitatively analyzed to obtain the pain detection result of the individual.

7. The pain detection method based on infrared imaging according to any one of claims 1 to 6, characterized in that: Also includes: Obtaining influencing factors that cause pain, and establishing a pain causal inference model based on the influencing factors; Obtaining the time corresponding to the pain detection result of the individual, and predicting the pain development trend of the individual based on the time; Evaluation indicators for the pain development trend are established through graph neural networks.

8. A pain detection device based on infrared imaging, characterized in that: The device comprises the following components: A preprocessing module, used for collecting original infrared images of individuals and preprocessing the original infrared images to obtain preprocessed infrared images; A labeling module, used for labeling the key parts on the preprocessed infrared image based on the semantics of the key parts of the individual, to obtain a semantically labeled infrared image; A segmentation module, used for segmenting the pain image corresponding to the key pain site on the semantically annotated infrared image; The detection module is used to obtain the body surface temperature of the individual at the key pain site, and obtain the pain detection result of the individual based on the pain image and the body surface temperature.

9. A terminal device, characterized in that: The terminal device includes a memory, a processor, and an infrared imaging-based pain detection program stored in the memory and executable on the processor. When the processor executes the infrared imaging-based pain detection program, the steps of the infrared imaging-based pain detection method as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a pain detection program based on infrared imaging. When the pain detection program based on infrared imaging is executed by a processor, the steps of the pain detection method based on infrared imaging as described in any one of claims 1-7 are implemented.

Images