Intelligent otolaryngology department operation pre-judgment system researched and developed based on patient behavior habits

Abstract

The invention discloses an otolaryngology department operation intelligent pre-judgment system researched and developed based on patient behavior habits, and belongs to the technical field of medical artificial intelligence. Comprising a first acquisition module, a second acquisition module, an image extraction module, a first target detection module, a first classification module, a second target detection module, a second classification module, an image sorting module, an early-stage correction module and an operation prognosis pre-judgment module. According to the method, the operation prognosis rehabilitation time is judged according to the behavior habits of the patient, the ways of doctors for judging the operation prognosis rehabilitation time of the patient are expanded, and the judgment accuracy is improved.

Description

Technical Field

[0001] The present invention belongs to the field of medical artificial intelligence technology, and specifically relates to an intelligent prediction system for otolaryngology surgery developed based on patient behavior habits. Background Art

[0002] ENT surgery is a common method for treating various ENT diseases, such as laryngeal tumors, vocal cord nodules and rhinitis. The recovery of wounds after surgery is not only related to the patient's recovery speed and effect, but also directly affects the patient's psychological state and quality of life. Therefore, determining the time of wound recovery after ENT surgery has important clinical significance.

[0003] The current method for determining the prognosis and recovery of ENT surgery mainly relies on the doctor's experience and professional knowledge, which is subjective and has certain limitations. In addition, since the prognosis and recovery of ENT surgery is easily affected by many factors, such as patient behavior habits, and doctors do not understand the patient's behavior habits, there are inevitably errors in the determination of recovery time, which can easily lead to medical disputes.

[0004] In view of this, an intelligent prediction system for otolaryngology surgery based on patients' behavioral habits was designed to predict the patients' recovery speed by predicting their degree of attention to otolaryngology prognosis based on their behavioral habits. Summary of the invention

[0005] To solve the problems raised in the above background technology, the present invention provides an intelligent prediction system for otolaryngology surgery developed based on patient behavior habits, which has the characteristics of judging the prognosis and recovery time of surgery based on the patient's behavior habits, expanding the way doctors judge the prognosis and recovery time of surgery for patients, and improving the accuracy of judgment.

[0006] To achieve the above-mentioned purpose, the present invention provides the following technical solutions: an intelligent prediction system for otolaryngology surgery developed based on patient behavior habits, comprising: a first acquisition module, a second acquisition module, an image extraction module, a first target detection module, a first classification module, a second target detection module, a second classification module, an image sorting module, an early correction module and a surgical prognosis prediction module, wherein:

[0007] The first collection module collects facial information and operation time information of ENT patients when they visit the hospital;

[0008] The second acquisition module captures the video stream in real time through a camera;

[0009] The image extraction module extracts images from the video stream frame by frame;

[0010] The first target detection module constructs and trains a first target detection model until convergence, and sequentially inputs the extracted images into the converged first target detection model, wherein the first target detection model detects images containing facial information of ENT patients based on the collected facial information of ENT patients and outputs the images to obtain mixed images of various ENT patients;

[0011] The first classification module constructs and trains a first classification model until convergence, and sequentially inputs the images output by the first target detection model into the converged first classification model, wherein the first classification model classifies and outputs the images based on the collected facial information of the otolaryngology patients, and obtains a series of images of each otolaryngology patient;

[0012] The second target detection module constructs and trains a second target detection model until convergence, and inputs the images output by the first classification model into the converged second target detection model in sequence according to the classification of otolaryngology patients. The second target detection model detects and outputs images containing behavioral habits that are not conducive to otolaryngology surgery rehabilitation based on pre-trained behavioral habits that are not conducive to otolaryngology surgery rehabilitation, thereby obtaining mixed images of behavioral habits that are not conducive to otolaryngology surgery rehabilitation for each otolaryngology patient;

[0013] The second classification module constructs and trains a second classification model until convergence, and inputs the images output by the second target detection model into the converged second classification model in sequence according to the classification of otolaryngology patients. The second classification model classifies and outputs the images based on a preset similarity, and obtains a series of images of behavioral habits of each otolaryngology patient that are not conducive to otolaryngology surgery rehabilitation;

[0014] The image sorting module sorts the series of images of the behavioral habits of each ENT patient that are not conducive to ENT surgery rehabilitation according to the timestamp;

[0015] The pre-correction module compares the behavioral habit series images of each ENT patient that are not conducive to ENT surgery rehabilitation with the collected operation time. If the behavioral habit series images of each ENT patient that are not conducive to ENT surgery rehabilitation are collected before the operation time, the behavioral habit series images of each ENT patient that are not conducive to ENT surgery rehabilitation are sent to the communication device of the attending physician, and the attending physician communicates and corrects them;

[0016] The surgical prognosis prediction module compares the series of images of behavioral habits of each ENT patient that are not conducive to ENT surgery rehabilitation with the collected operation time. If the series of images of behavioral habits of each ENT patient that are not conducive to ENT surgery rehabilitation are collected after the operation time, the similarity of the series of images of behavioral habits of each ENT patient that are not conducive to ENT surgery rehabilitation after the operation is compared with that before the operation, and the rehabilitation cooperation of each ENT patient is determined based on the similarity, and the surgical prognosis recovery time of each ENT patient is determined based on the rehabilitation cooperation.

[0017] Furthermore, the first acquisition module acquires facial information of ENT patients, including acquiring facial information through a high-definition camera when the ENT patients visit the doctor.

[0018] Furthermore, the specific steps of the similarity comparison of the surgical prognosis prediction module include:

[0019] S101: Processing the behavioral habit images of ENT patients before and after surgery that are not conducive to ENT surgery recovery through a filter to extract contours, wherein the expression of the filter is:

[0020]

[0021] Where: k represents the total number of directions, v represents the filtering wavelength, μ represents the parallel strip direction, e represents the filtering amplitude, z represents the image coordinate, σ represents the standard deviation of the Gaussian factor, and i represents the oscillation phase; It is expressed as phase;

[0022] S102: setting a pixel gray value threshold, defining the pixel gray value exceeding the preset threshold in the image of the behavior habits of the otolaryngology patient before and after surgery that are not conducive to the recovery of otolaryngology surgery as 1, and vice versa as 0, and performing binarization;

[0023] S103: traversing pixels of the behavioral habit images of ENT patients before and after surgery that are not conducive to ENT surgery recovery, and performing full associative mapping on pixels with grayscale values ​​of 0 to obtain fingerprints;

[0024] S104: Perform a fully connected mapping of the fingerprints of the behavioral habit images of ENT patients before and after surgery that are not conducive to ENT surgery recovery as a quotient. If the distance between the quotient value and 1 exceeds a set threshold, it is defined as 1, otherwise it is positioned as 0. The Hamming distance is determined based on the number composed of 1 and 0. If it is less than the preset threshold, it is determined to be similar.

[0025] Furthermore, the specific steps of judging the degree of rehabilitation cooperation of the surgical prognosis prediction module include:

[0026] S105: Presetting a matching rule between similarity and rehabilitation coordination;

[0027] S106: Determine the rehabilitation compliance based on the determined similarity.

[0028] Furthermore, the specific steps of determining the surgical prognosis recovery time of ENT patients in the surgical prognosis prediction module include:

[0029] S107: collecting historical prognosis data of patients undergoing ENT surgery, wherein the historical prognosis data of patients undergoing ENT surgery includes data of patients without behavioral habits that are not conducive to recovery from ENT surgery and data of patients with behavioral habits that are not conducive to recovery from ENT surgery;

[0030] S108: Analysis of the recovery time for each prognosis of ENT surgery based on the data of patients who have no behavioral habits that are not conducive to ENT surgery recovery;

[0031] S109: Classifying the data of patients with behavioral habits that are not conducive to ENT surgery recovery based on the behavioral habits, and classifying the data of patients with behavioral habits that are not conducive to ENT surgery recovery based on the severity of the same behavioral habits;

[0032] S110: Analyze the influence of each behavioral habit that is not conducive to the recovery of ENT surgery on the patient's surgical prognosis and recovery based on the data of patients with behavioral habits that are not conducive to the recovery of ENT surgery after the first classification;

[0033] S111: Based on the data of patients with behavioral habits that are not conducive to ENT surgery rehabilitation after secondary classification, analyze the influence of different degrees of behavioral habits that are not conducive to ENT surgery rehabilitation on the patient's surgical prognosis and rehabilitation;

[0034] S112: Determine the surgical prognosis recovery time of the ENT patient based on the degree of rehabilitation cooperation, various behavioral habits that are not conducive to ENT surgery rehabilitation, and the degree of influence of various degrees of behavioral habits that are not conducive to ENT surgery rehabilitation on the patient's surgical prognosis recovery.

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] The present invention collects facial information and operation time information of ENT surgery patients in real time when they visit the doctor. During the hospitalization period from before to after the ENT surgery, the camera collects video streams in real time and extracts images frame by frame. A target detection model and a classification model are used to extract a series of images of behavioral habits of each ENT patient that are not conducive to ENT surgery rehabilitation based on the collected facial information, and the images are sorted. Then, based on the collected operation time information, the period to which the images belong is determined. If the images are before the surgery, they are sent to the attending physician so that the attending physician can communicate and correct them. If the images are after the surgery, the patient's rehabilitation cooperation is determined based on the comparison of pre- and post-operative images. The patient's surgical prognosis rehabilitation time is determined based on the rehabilitation cooperation, behavioral habits that are not conducive to ENT surgery rehabilitation, and the degree. Compared with the prior art, this method judges the surgical prognosis rehabilitation time based on the patient's behavioral habits, expands the doctor's way of judging the patient's surgical prognosis rehabilitation time, and improves the accuracy of the judgment. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is a system framework diagram of the present invention;

[0038] In the figure: 1. First acquisition module; 2. Second acquisition module; 3. Image extraction module; 4. First target detection module; 5. First classification module; 6. Second target detection module; 7. Second classification module; 8. Image sorting module; 9. Preliminary correction module; 10. Post-operative prognosis prediction module. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] See also Figure 1 The present invention provides the following technical solution: an intelligent prediction system for otolaryngology surgery developed based on patient behavior habits, comprising: a first acquisition module 1, a second acquisition module 2, an image extraction module 3, a first target detection module 4, a first classification module 5, a second target detection module 6, a second classification module 7, an image sorting module 8, a pre-correction module 9 and a surgery prognosis prediction module 10, wherein:

[0041] The first collection module 1 collects facial information and operation time information of ENT patients when they visit the hospital;

[0042] The first acquisition module 1 acquires facial information of ENT patients, including acquiring facial information through a high-definition camera when the ENT patients visit the doctor;

[0043] When an ENT patient visits a doctor, facial information is collected through a high-definition camera, which requires the patient's consent and authorization. If the patient does not agree to the authorization, the facial information collection will be interrupted and the system will not be able to provide a prediction. This point needs to be communicated with the patient so that the patient can understand it.

[0044] The second acquisition module 2 captures the video stream in real time through a camera;

[0045] Image extraction module 3, extracting images of the video stream frame by frame;

[0046] The first target detection module 4 constructs and trains a first target detection model until convergence, and sequentially inputs the extracted images into the converged first target detection model. The first target detection model detects images containing facial information of ENT patients based on the collected facial information of ENT patients and outputs them, thereby obtaining mixed images of various ENT patients;

[0047] The first target detection model is the YOLOv5 target detection model;

[0048] The first classification module 5 constructs and trains the first classification model until convergence, and sequentially inputs the images output by the first target detection model into the converged first classification model. The first classification model classifies and outputs the images based on the collected facial information of the otolaryngology patients, and obtains a series of images of each otolaryngology patient;

[0049] The first classification model is the ResNet classification model;

[0050] The second target detection module 6 constructs and trains a second target detection model until convergence, and inputs the images output by the first classification model into the converged second target detection model in sequence according to the classification of otolaryngology patients. The second target detection model detects images containing behavioral habits that are not conducive to otolaryngology surgery rehabilitation based on pre-trained behavioral habits that are not conducive to otolaryngology surgery rehabilitation and outputs them, thereby obtaining mixed images of behavioral habits that are not conducive to otolaryngology surgery rehabilitation for each otolaryngology patient;

[0051] The second target detection model is the YOLOv5 target detection model;

[0052] The second classification module 7 constructs and trains the second classification model until convergence, and inputs the images output by the second target detection model into the converged second classification model in sequence according to the classification of otolaryngology patients. The second classification model classifies and outputs the images based on the preset similarity, and obtains a series of images of the behavioral habits of each otolaryngology patient that are not conducive to otolaryngology surgery rehabilitation;

[0053] The second classification model is the ResNet classification model;

[0054] An image sorting module 8, sorting the series of images of the behavioral habits of each ENT patient that are not conducive to ENT surgery recovery according to the timestamp;

[0055] The pre-correction module 9 compares the behavioral habit series images of each ENT patient that are not conducive to ENT surgery recovery with the collected operation time. If the behavioral habit series images of each ENT patient that are not conducive to ENT surgery recovery are collected before the operation time, the behavioral habit series images of each ENT patient that are not conducive to ENT surgery recovery are sent to the communication device of the attending physician, and the attending physician communicates and corrects them;

[0056] The surgical prognosis prediction module 10 compares the behavioral habit series images of each ENT patient that are not conducive to ENT surgery rehabilitation with the collected surgery time. If the behavioral habit series images of each ENT patient that are not conducive to ENT surgery rehabilitation are collected after the surgery time, compare the similarity of the behavioral habit series images of each ENT patient that are not conducive to ENT surgery rehabilitation after the surgery and before the surgery, determine the rehabilitation cooperation of each ENT patient based on the similarity, and determine the surgical prognosis rehabilitation time of each ENT patient based on the rehabilitation cooperation;

[0057] The specific steps of the similarity comparison of the surgical prognosis prediction module 10 include:

[0058] S101: Processing the behavioral habit images of ENT patients before and after surgery that are not conducive to ENT surgery recovery through a filter to extract contours, wherein the expression of the filter is:

[0059]

[0060] Where: k represents the total number of directions, v represents the filtering wavelength, μ represents the parallel strip direction, e represents the filtering amplitude, z represents the image coordinate, σ represents the standard deviation of the Gaussian factor, and i represents the oscillation phase; It is expressed as phase;

[0061] S102: setting a pixel gray value threshold, defining the pixel gray value exceeding the preset threshold in the image of the behavior habits of the otolaryngology patient before and after surgery that are not conducive to the recovery of otolaryngology surgery as 1, and vice versa as 0, and performing binarization;

[0062] S103: traversing pixels of the behavioral habit images of ENT patients before and after surgery that are not conducive to ENT surgery recovery, and performing full associative mapping on pixels with grayscale values ​​of 0 to obtain fingerprints;

[0063] S104: Perform a fully connected mapping of the fingerprints of the behavioral habit images of ENT patients before and after surgery that are not conducive to ENT surgery recovery as a quotient. If the distance between the quotient value and 1 exceeds a set threshold, it is defined as 1, otherwise it is positioned as 0. The Hamming distance is determined based on the number composed of 1 and 0. If it is less than the preset threshold, it is determined to be similar.

[0064] The specific steps of judging the degree of rehabilitation cooperation of the surgical prognosis prediction module 10 include:

[0065] S105: Presetting a matching rule between similarity and rehabilitation coordination;

[0066] S106: Determine the rehabilitation compliance based on the determined similarity.

[0067] The specific steps of determining the surgical prognosis recovery time of ENT patients in the surgical prognosis prediction module 10 include:

[0068] S107: collecting historical prognosis data of patients undergoing ENT surgery, wherein the historical prognosis data of patients undergoing ENT surgery includes data of patients without behavioral habits that are not conducive to recovery from ENT surgery and data of patients with behavioral habits that are not conducive to recovery from ENT surgery;

[0069] S108: Analysis of the recovery time for each prognosis of ENT surgery based on the data of patients who have no behavioral habits that are not conducive to ENT surgery recovery;

[0070] S109: Classifying the data of patients with behavioral habits that are not conducive to ENT surgery recovery based on the behavioral habits, and classifying the data of patients with behavioral habits that are not conducive to ENT surgery recovery based on the severity of the same behavioral habits;

[0071] S110: Analyze the influence of each behavioral habit that is not conducive to the recovery of ENT surgery on the patient's surgical prognosis and recovery based on the data of patients with behavioral habits that are not conducive to the recovery of ENT surgery after the first classification;

[0072] S111: Based on the data of patients with behavioral habits that are not conducive to ENT surgery rehabilitation after secondary classification, analyze the influence of different degrees of behavioral habits that are not conducive to ENT surgery rehabilitation on the patient's surgical prognosis and rehabilitation;

[0073] S112: Determine the surgical prognosis recovery time of the ENT patient based on the degree of rehabilitation cooperation, various behavioral habits that are not conducive to ENT surgery rehabilitation, and the degree of influence of various degrees of behavioral habits that are not conducive to ENT surgery rehabilitation on the patient's surgical prognosis recovery.

[0074] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An intelligent prediction system for otolaryngology surgery developed based on patient behavior habits, characterized in that: include: A first acquisition module (1), a second acquisition module (2), an image extraction module (3), a first target detection module (4), a first classification module (5), a second target detection module (6), a second classification module (7), an image sorting module (8), an early correction module (9) and a post-operative prognosis prediction module (10), wherein: The first collection module (1) collects facial information and operation time information of ENT patients when they visit the doctor; The second acquisition module (2) captures the video stream in real time through a camera; The image extraction module (3) extracts images from the video stream frame by frame; The first target detection module (4) constructs and trains a first target detection model until convergence, and sequentially inputs the extracted images into the converged first target detection model. The first target detection model detects images containing facial information of otolaryngology patients based on the collected facial information of otolaryngology patients and outputs the images, thereby obtaining mixed images of various otolaryngology patients; The first classification module (5) constructs and trains a first classification model until convergence, and sequentially inputs the images output by the first target detection model into the converged first classification model, wherein the first classification model classifies and outputs the images based on the collected facial information of the otolaryngology patients, thereby obtaining a series of images of each otolaryngology patient; The second target detection module (6) constructs and trains a second target detection model until convergence, and inputs the images output by the first classification model into the converged second target detection model in sequence according to the classification of otolaryngology patients. The second target detection model detects images containing behavioral habits that are not conducive to otolaryngology surgery rehabilitation based on pre-trained behavioral habits that are not conducive to otolaryngology surgery rehabilitation and outputs them, thereby obtaining mixed images of behavioral habits that are not conducive to otolaryngology surgery rehabilitation for each otolaryngology patient; The second classification module (7) constructs and trains a second classification model until convergence, and inputs the images output by the second target detection model into the converged second classification model in sequence according to the classification of otolaryngology patients. The second classification model classifies and outputs the images based on a preset similarity, thereby obtaining a series of images of behavioral habits of each otolaryngology patient that are not conducive to otolaryngology surgery rehabilitation; The image sorting module (8) sorts the series of images of the behavioral habits of each ENT patient that are not conducive to ENT surgery recovery according to the timestamp; The pre-correction module (9) compares the series of images of the behavioral habits of each ENT patient that are not conducive to ENT surgery rehabilitation with the collected operation time. If the series of images of the behavioral habits of each ENT patient that are not conducive to ENT surgery rehabilitation are collected before the operation time, the series of images of the behavioral habits of each ENT patient that are not conducive to ENT surgery rehabilitation are sent to the communication device of the attending physician, and the attending physician communicates and corrects them; The surgical prognosis prediction module (10) compares the behavioral habit series images of each ENT patient that are not conducive to ENT surgery rehabilitation with the collected surgery time. If the behavioral habit series images of each ENT patient that are not conducive to ENT surgery rehabilitation are collected after the surgery time, the similarity of the behavioral habit series images of each ENT patient that are not conducive to ENT surgery rehabilitation after the surgery and before the surgery is compared, and the rehabilitation cooperation of each ENT patient is determined based on the similarity, and the surgical prognosis rehabilitation time of each ENT patient is determined based on the rehabilitation cooperation.

2. According to claim 1, an intelligent prediction system for ENT surgery developed based on patient behavior habits is characterized by: The first acquisition module (1) acquires facial information of ENT patients, including acquiring facial information through a high-definition camera when the ENT patients visit the doctor.

3. According to claim 1, an intelligent prediction system for ENT surgery developed based on patient behavior habits is characterized by: The specific steps of the similarity comparison of the surgical prognosis prediction module (10) include: S101: Processing the behavioral habit images of ENT patients before and after surgery that are not conducive to ENT surgery recovery through a filter to extract contours, wherein the expression of the filter is: Where: k represents the total number of directions, v represents the filtering wavelength, μ represents the parallel strip direction, e represents the filtering amplitude, z represents the image coordinate, σ represents the standard deviation of the Gaussian factor, and i represents the oscillation phase; It is expressed as phase; S102: setting a pixel gray value threshold, defining the pixel gray value exceeding the preset threshold in the image of the behavior habits of the otolaryngology patient before and after surgery that are not conducive to the recovery of otolaryngology surgery as 1, and vice versa as 0, and performing binarization; S103: traversing pixels of the behavioral habit images of ENT patients before and after surgery that are not conducive to ENT surgery recovery, and performing full associative mapping on pixels with grayscale values ​​of 0 to obtain fingerprints; S104: Perform a fully connected mapping of the fingerprints of the behavioral habit images of ENT patients before and after surgery that are not conducive to ENT surgery recovery as a quotient. If the distance between the quotient value and 1 exceeds a set threshold, it is defined as 1, otherwise it is positioned as 0. The Hamming distance is determined based on the number composed of 1 and 0. If it is less than the preset threshold, it is determined to be similar.

4. According to claim 1, an intelligent prediction system for ENT surgery developed based on patient behavior habits is characterized by: The specific steps of judging the degree of rehabilitation compliance of the surgical prognosis prediction module (10) include: S105: Presetting a matching rule between similarity and rehabilitation coordination; S106: Determine the rehabilitation compliance based on the determined similarity.

5. According to claim 1, an intelligent prediction system for ENT surgery developed based on patient behavior habits is characterized by: The specific steps of determining the surgical prognosis recovery time of an otolaryngology patient in the surgical prognosis prediction module (10) include: S107: collecting historical prognosis data of patients undergoing ENT surgery, wherein the historical prognosis data of patients undergoing ENT surgery includes data of patients without behavioral habits that are not conducive to recovery from ENT surgery and data of patients with behavioral habits that are not conducive to recovery from ENT surgery; S108: Analysis of the recovery time for each prognosis of ENT surgery based on the data of patients who have no behavioral habits that are not conducive to ENT surgery recovery; S109: Classifying the data of patients with behavioral habits that are not conducive to ENT surgery recovery based on the behavioral habits, and classifying the data of patients with behavioral habits that are not conducive to ENT surgery recovery based on the severity of the same behavioral habits; S110: Analyze the influence of each behavioral habit that is not conducive to the recovery of ENT surgery on the patient's surgical prognosis and recovery based on the data of patients with behavioral habits that are not conducive to the recovery of ENT surgery after the first classification; S111: Based on the data of patients with behavioral habits that are not conducive to ENT surgery rehabilitation after secondary classification, analyze the influence of different degrees of behavioral habits that are not conducive to ENT surgery rehabilitation on the patient's surgical prognosis and rehabilitation; S112: Determine the surgical prognosis recovery time of the ENT patient based on the degree of rehabilitation cooperation, various behavioral habits that are not conducive to ENT surgery rehabilitation, and the degree of influence of various degrees of behavioral habits that are not conducive to ENT surgery rehabilitation on the patient's surgical prognosis recovery.

Images