Accompanying diagnosis management system based on artificial intelligence technology

Abstract

The invention provides an accompanying diagnosis management system based on an artificial intelligence technology. The accompanying diagnosis management system comprises a wearable acquisition terminal and an intelligent analysis management platform, wherein the wearable acquisition terminal is worn on an accompanying doctor and is used for acquiring accompanying diagnosis video data recorded in an accompanying diagnosis process and uploading the accompanying diagnosis video data to the intelligent analysis management platform; the intelligent analysis management platform is used for extracting corresponding scene features and accompanying diagnosis features according to the acquired accompanying diagnosis video data, accompanying diagnosis performance analysis is carried out by adopting a trained analysis model according to the acquired scene features and accompanying diagnosis features, an accompanying diagnosis performance analysis result is obtained, and the accompanying diagnosis features comprise behavior features and / or speech features. According to the invention, the quality and the management level of the diagnosis accompanying service can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of accompanying medical consultation management, and in particular to an accompanying medical consultation management system based on artificial intelligence technology. Background Art

[0002] With the increasing level of hospital construction, the process of patients seeking medical treatment in hospitals has become more and more convenient. However, for special groups such as the elderly, pregnant women, and minors, they still encounter many problems when seeking medical treatment in hospitals, so they usually need to be accompanied by family members or relatives to complete the medical process. However, since the cooperation of peers is required, the difficulty of seeking medical treatment for special groups has been greatly increased. In this regard, the emergence of accompanying doctors can largely meet the needs of such special groups in the process. By accompanying patients in the hospital, accompanying doctors can assist patients in various procedural tasks required during the medical treatment process, guide and assist patients to complete the entire medical process, which can save patients a lot of time and energy and improve patients' medical experience.

[0003] At present, due to the lack of systematic management methods, most of the accompanying doctors provide services in an individual manner. Therefore, the service quality and qualifications of the accompanying doctors are uneven, which is not conducive to the development and management of the accompanying doctors industry personnel. Summary of the invention

[0004] In view of the above problems, the present invention aims to provide a medical companion management system based on artificial intelligence technology.

[0005] The purpose of the present invention is achieved by adopting the following technical solutions:

[0006] The present invention shows a medical companion management system based on artificial intelligence technology, which is characterized by comprising a wearable acquisition terminal and an intelligent analysis management platform; wherein,

[0007] The wearable data collection terminal is worn on the accompanying physician to collect the accompanying video data recorded during the accompanying process and upload the accompanying video data to the intelligent analysis management platform;

[0008] The intelligent analysis management platform is used to extract corresponding scene features and accompanying medical features based on the acquired accompanying medical video data, and use the trained analysis model to perform accompanying medical performance analysis based on the acquired scene features and accompanying medical features to obtain accompanying medical performance analysis results, wherein the accompanying medical features include behavioral features and / or speech features.

[0009] Preferably, the wearable acquisition terminal includes a video recording module and a wireless transmission module; wherein,

[0010] The video recording module is used to collect the accompanying video data of the accompanying doctor during the accompanying process, wherein the accompanying video data includes video picture information and voice information;

[0011] The wireless transmission module is used to transmit the acquired accompanying medical video data to the intelligent analysis and management platform in real time.

[0012] Preferably, the intelligent analysis management platform includes a scene extraction module, a medical companion extraction module, a medical companion analysis module and a result output module; wherein,

[0013] The scene extraction module is used to perform scene analysis based on the video picture information of the acquired accompanying video data, and obtain the scene features corresponding to the current accompanying video data, wherein the scene features include the location information of the current accompanying video data, including wards, consulting rooms, treatment rooms, examination rooms, hospital lobbies, pharmacies, registration offices, cashier offices, etc.;

[0014] The accompanying diagnosis extraction module is used to extract behavior features based on the video picture information of the accompanying diagnosis video data obtained to obtain corresponding behavior features; and / or to extract speech features based on the voice information of the accompanying diagnosis video data obtained to obtain corresponding speech features; and to form accompanying diagnosis features based on the obtained behavior features and / or speech features;

[0015] The accompanying diagnosis analysis module is used to input the acquired scene features and accompanying diagnosis features into the trained analysis model, and the analysis model performs accompanying diagnosis performance analysis on the corresponding accompanying diagnosis features according to the scene features to obtain accompanying diagnosis performance analysis results; wherein the accompanying diagnosis performance analysis results include accompanying diagnosis performance rating information;

[0016] The result output module is used to output the corresponding accompanying performance analysis results, and when the accompanying performance analysis results are abnormal, output the corresponding warning message.

[0017] Preferably, the intelligent analysis management platform also includes a preprocessing module;

[0018] The preprocessing module is used to first preprocess the received accompanying medical consultation video data to improve the quality of the accompanying medical consultation video data, and further transmit the preprocessed accompanying medical consultation video data to the scene extraction module and / or the accompanying medical consultation extraction module.

[0019] Preferably, the scene extraction module specifically includes:

[0020] Performing scene recognition based on the video picture information of the acquired accompanying medical video data, including detecting scene markers in the video picture information, and determining the scene features of the current video picture information based on the detected scene markers;

[0021] The scene markers include signboards, medical equipment, medical staff, etc., and each scene marker corresponds to one or more location information.

[0022] Preferably, the accompanying diagnosis extraction module includes a behavior extraction unit and / or a speech extraction unit; wherein,

[0023] The behavior extraction unit is used to extract behavior features according to the video picture information of the acquired accompanying medical video data, including extracting the hand behavior movements of the current accompanying medical practitioner according to the video picture information, and obtaining corresponding behavior features according to the hand behavior movements, wherein the behavior features include operating a mobile phone, picking up objects, and human contact, etc.;

[0024] The speech extraction unit is used to extract speech features based on the speech information of the acquired accompanying medical consultation video data, including performing speech recognition processing based on the language information, extracting corresponding text information, and obtaining corresponding speech features after cleaning and standardization processing based on the extracted text information, wherein the speech features include the speech text information of the accompanying medical consultation physician and other personnel.

[0025] Preferably, the accompanying diagnosis analysis module includes: forming an input vector according to each time feature and the scene feature, behavior feature and speech feature corresponding to the same time, and inputting the input vector into a trained analysis model, and the analysis model performs accompanying diagnosis performance analysis on the corresponding accompanying diagnosis features according to the scene features to obtain accompanying diagnosis performance analysis results.

[0026] Preferably, the accompanying diagnosis analysis module includes a model training unit;

[0027] The model training unit is used to train the analysis model, wherein the analysis model is constructed by using a CNN convolutional neural network based on deep learning technology, wherein the analysis model includes an input layer, a hidden layer and an output layer, wherein the input layer is used to input a constructed input vector, and the hidden layer performs convolution processing on the input data to extract data features, and the output layer sets a Softmax classifier to classify the convolution features to obtain corresponding accompanying medical performance analysis results; wherein in the process of training the analysis model, corresponding environmental features and accompanying medical features are extracted based on pre-prepared standard accompanying medical video data, a training set is formed based on environmental features, accompanying medical features and corresponding calibrated standard performance analysis results, and the analysis model is trained based on the training set until the training model guarantees a certain accuracy rate when testing the test set, thereby obtaining a trained analysis model.

[0028] Preferably, the intelligent analysis management platform also includes one or more of a playback module, a data management module, a call module and a medical companion file module; wherein,

[0029] The playback module is used to play the accompanying video data transmitted back by the wearable acquisition terminal in real time;

[0030] The call module is used to establish a real-time call connection with the wearable acquisition terminal;

[0031] The data management module is used to perform privacy protection processing on the received accompanying medical video data, and to classify, store and manage the accompanying medical video data after the privacy protection processing, and to build an accompanying medical video database;

[0032] The medical companion file module is used to generate a medical companion file based on the medical companion video data and the corresponding medical companion performance analysis results transmitted by the medical companion.

[0033] The beneficial effects of the present invention are as follows: by wearing an image acquisition device on the accompanying attendant to record the whole process information of the accompanying attendant in the process of completing the accompanying service for the patient, and by using an intelligent analysis management platform to perform real-time analysis based on the recorded data, real-time evaluation and management of the accompanying service are provided, which helps managers to provide real-time guidance to the accompanying attendant based on the accompanying attendant's recorded behavior and performance analysis results, or allows the accompanying attendant to understand and review his or her own accompanying service, thereby improving the quality and management level of the accompanying service, and also helps to improve the patient's medical experience and promote the improvement of medical services. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The present invention is further described using the accompanying drawings, but the embodiments in the accompanying drawings do not constitute any limitation to the present invention. A person skilled in the art can obtain other drawings based on the following drawings without creative work.

[0035] Figure 1 This is a framework diagram of a medical companion management system based on artificial intelligence technology as shown in an embodiment of the present invention;

[0036] Figure 2 This is a module structure diagram of the intelligent analysis management platform shown in an embodiment of the present invention. DETAILED DESCRIPTION

[0037] The present invention is further described in conjunction with the following application scenarios.

[0038] See also Figure 1 , Figure 2 The embodiment shows an accompanying medical treatment management system based on artificial intelligence technology, including a wearable collection terminal and an intelligent analysis management platform; wherein,

[0039] The wearable data collection terminal is worn on the accompanying physician to collect the accompanying video data recorded during the accompanying process and upload the accompanying video data to the intelligent analysis management platform;

[0040] The intelligent analysis management platform is used to extract corresponding scene features and accompanying medical features based on the acquired accompanying medical video data, and use the trained analysis model to perform accompanying medical performance analysis based on the acquired scene features and accompanying medical features to obtain accompanying medical performance analysis results, wherein the accompanying medical features include behavioral features and / or speech features.

[0041] The above-mentioned implementation mode of the present invention records the whole process information of the accompanying therapist in the process of completing the accompanying service for the patient by wearing an image acquisition device on the accompanying therapist (such as on clothes, hat, etc.), and performs real-time analysis based on the recorded data through an intelligent analysis management platform to provide real-time evaluation and management of the accompanying service. It helps managers to provide real-time guidance to the accompanying therapist based on the accompanying therapist's recorded behavior and performance analysis results, or allows the accompanying therapist to understand and review his or her own accompanying service, thereby improving the quality and management level of the accompanying service, and also helps to improve the patient's medical experience and promote the improvement of medical services.

[0042] Preferably, the wearable acquisition terminal includes a video recording module and a wireless transmission module; wherein,

[0043] The video recording module is used to collect the accompanying video data of the accompanying doctor during the accompanying process, wherein the accompanying video data includes video picture information and voice information;

[0044] The wireless transmission module is used to transmit the acquired accompanying medical video data to the intelligent analysis and management platform in real time.

[0045] In one scenario, a camera is installed on the wearable acquisition terminal. The camera is worn on the clothes of the accompanying doctor to record the whole process of the patient's visit. The camera records the video data and audio data in front of the accompanying doctor, and transmits the collected data back to the intelligent analysis management platform in real time by wireless transmission for further data processing. Through the collection of accompanying video data, the situation of the accompanying doctor and the patient during the accompanying service can be truly reflected. At the same time, using video data as the basis for subsequent analysis can also help capture more information on the service site and provide accurate analysis and management for the accompanying service.

[0046] In one scenario, the wireless transmission module can transmit the acquired accompanying medical video data to the intelligent analysis and management platform in real time through WIFI, 5G and other means.

[0047] Preferably, the intelligent analysis management platform includes a scene extraction module, a medical companion extraction module, a medical companion analysis module and a result output module; wherein,

[0048] The scene extraction module is used to perform scene analysis based on the video picture information of the acquired accompanying video data, and obtain the scene features corresponding to the current accompanying video data, wherein the scene features include the location information of the current accompanying video data, including wards, consulting rooms, treatment rooms, examination rooms, hospital lobbies, pharmacies, registration offices, cashier offices, etc.;

[0049] The accompanying diagnosis extraction module is used to extract behavior features based on the video picture information of the accompanying diagnosis video data obtained to obtain corresponding behavior features; and / or to extract speech features based on the voice information of the accompanying diagnosis video data obtained to obtain corresponding speech features; and to form accompanying diagnosis features based on the obtained behavior features and / or speech features;

[0050] The accompanying diagnosis analysis module is used to input the acquired scene features and accompanying diagnosis features into the trained analysis model, and the analysis model performs accompanying diagnosis performance analysis on the corresponding accompanying diagnosis features according to the scene features to obtain accompanying diagnosis performance analysis results; wherein the accompanying diagnosis performance analysis results include accompanying diagnosis performance rating information;

[0051] The result output module is used to output the corresponding accompanying performance analysis results, and when the accompanying performance analysis results are abnormal, output the corresponding warning message.

[0052] Among them, the intelligent analysis management platform can be built based on a server set up locally in the hospital or based on a cloud server. Based on the built intelligent analysis management platform, it can centrally analyze and record the accompanying video data collected by each accompanying physician during the accompanying service in real time, providing a basis for data management and data support.

[0053] In one scenario, when analyzing the acquired medical accompanying video data, the current scene location is first identified based on the environmental features in the video, and the medical accompanying behavior features and speech features of the medical accompanying therapist are further extracted according to the content of the video data. A feature set is formed according to the obtained environmental features and medical accompanying features, and the feature set is analyzed and processed based on the trained analysis model to obtain the medical accompanying performance analysis results. The obtained medical accompanying performance analysis results are returned to the manager and the medical accompanying therapist himself, so that the manager and the medical accompanying therapist can timely understand the current medical accompanying service situation, make timely guidance and corrections, thereby helping to guide the improvement of the quality of medical accompanying services.

[0054] Preferably, the intelligent analysis management platform also includes a preprocessing module;

[0055] The preprocessing module is used to first preprocess the received accompanying medical consultation video data to improve the quality of the accompanying medical consultation video data, and further transmit the preprocessed accompanying medical consultation video data to the scene extraction module and / or the accompanying medical consultation extraction module.

[0056] Considering the video data of accompanying patients collected in the actual hospital scene, since the accompanying doctors are usually in a non-static state during the data collection process, the scene of the collected accompanying video data also changes accordingly. The lighting environment in the hospital is usually white light, which is not natural light. Therefore, in the actual video footage, it is easy to have backlight or white light directly shining on the screen, causing serious exposure, which affects the clarity and quality of the accompanying video, thereby affecting the effect of further playing the accompanying video and the accuracy and reliability of extracting environmental features and behavioral features based on the accompanying video.

[0057] Therefore, the above implementation also proposes to pre-process the collected medical video data by setting a pre-processing module on the intelligent analysis management platform, so as to improve the quality of the video image, thereby indirectly improving the effect of subsequent medical video image playback and intelligent analysis.

[0058] Preferably, the preprocessing module includes a picture enhancement unit; the picture enhancement unit is used to enhance the video picture of the received accompanying medical video data to improve the picture quality of the accompanying medical video data, specifically including:

[0059] Extract the video screen of the accompanying medical consultation according to the acquired accompanying medical consultation video data, and further use the Sobel edge detection operator to detect the edge pixels in the accompanying medical consultation video screen, mark the detected edge pixels as Q(x,y)=B, and mark the non-edge pixels as Q(x,y)=C, where Q(x,y) represents the classification label of the pixel at the pixel position (x,y);

[0060] The video screen of the accompanying medical consultation is divided into grids according to the detected edge pixels, including:

[0061] Initialize the division factor i=1; in the first round of grid division, use the accompanying video screen as the target grid, and divide the target grid into 2×2 sub-grids of the same size;

[0062] For each sub-grid obtained, a judgment is made. When the number of edge pixels in the sub-grid Num(i,B) ≥ α i ×Num(i), then the subgrid is marked as a feature grid, otherwise the grid is marked as a general grid; where α i represents the proportional adjustment factor, IN represents the maximum number of division rounds, k1 and k2 represent the set proportional control factors, and Num(i) represents the total number of pixels in the current sub-grid;

[0063] Start the next round of grid division, adjust the grid division factor i=i+1, and perform the i-th round of grid division for each feature grid marked in the previous round, wherein the feature grid is used as the target grid and the target grid is divided into 2×2 sub-grids of the same size;

[0064] For each sub-grid obtained, a judgment is made. When the number of edge pixels in the sub-grid Num(i,B) ≥ α i ×Num(i), then the sub-grid is marked as a feature grid, otherwise the grid is marked as a general grid;

[0065] Repeat the above method to perform mesh division until the mesh division factor i≥IN, or the length and width of the sub-grid are smaller than the preset minimum mesh size value LiH, then stop mesh division and obtain the final mesh division result;

[0066] Based on the HSV color space, the hue component H, saturation component S and brightness component V of the accompanying medical video are extracted respectively;

[0067] Calculate the Gaussian eigenvalue of each pixel based on the obtained brightness component:

[0068]

[0069] Where Gors(x,y) represents the Gaussian eigenvalue at the pixel position (x,y), V(x,y) represents the brightness component value at the pixel position (x,y), and n represents the function variable, where n = 1, 2, ... N, N is the total number of kernel scales set, 3 ≤ N ≤ 5, σ n represents the nth Gaussian kernel scale factor, where σ n =N×3 n-1 ;

[0070] Calculate the brightness eigenvalue of each sub-grid separately:

[0071]

[0072] Where VLT(k) represents the brightness feature value of the kth subgrid, where the variable k = 1, 2, ... K, K represents the total number of subgrids, Num(k) represents the total number of pixels in the kth subgrid, (x, y)∈k represents the variable (x, y) is a pixel in the kth subgrid, hi(k) and lo(k) represent the height and width of the kth subgrid, respectively, and Dist to-a(k) (x,y) represents the distance from the pixel position (x,y) to the center point position a(k) of the kth subgrid, Gors(x,y) represents the Gaussian eigenvalue at the pixel position (x,y);

[0073] Analyze the brightness characteristic values ​​of each sub-grid:

[0074] Sort the sub - grids in ascending order according to their brightness characteristic values, and start detecting the sub - grids in ascending order from the \(j = 1\)st sub - grid after sorting. When the \(j\)th sub - grid is detected to meet \(VLT(j)<THVLT1\) and \(|VLT(j)-VLT(j + 2)|\geq THVLT2\), mark the \(j\)th sub - grid as the first characteristic region, and further update \(j=j + 1\). Repeat the above detection until all sub - grids are detected; where \(THVLT1\) represents the set first judgment threshold, and \(THVLT2\) represents the set second judgment threshold;

[0075] Sort the sub - grids in descending order according to their brightness characteristic values, and start detecting the sub - grids in order from the \(j = 1\)st sub - grid after sorting. When the \(j\)th sub - grid is detected to meet \(VLT()\geq THVLT3\) and \(|VLT(j)-VLT(j + 2)|\geq THVLT4\), mark the \(j\)th sub - grid as the second characteristic region, and further update \(j=j + 1\). Repeat the above detection until all sub - grids are detected; where \(THVLT3\) represents the set third judgment threshold, and \(THVLT4\) represents the set fourth judgment threshold;

[0076] And mark the remaining regions that do not belong to the first and second characteristic regions as the third characteristic region;

[0077] Perform brightness enhancement processing on the third characteristic region:

[0078] \(V′3(x,y)=0.7\times V(x,y)+0.3\times THV\)

[0079] In the formula, the variable \((x,y)\) represents the pixel points belonging to the third characteristic region, \(V'3(x,y)\) represents the brightness component value at the pixel point position \((x,y)\) after brightness enhancement processing, \(V(x,y)\) represents the brightness component value at the pixel point position \((x,y)\), and \(THV\) represents the set standard brightness component value, where \(0.7\leq THV\leq0.75\);

[0080] Perform brightness enhancement processing on the first characteristic region:

[0081]

[0082] In the formula, the variable \((x,y)\) represents the pixel points belonging to the first characteristic region, \(V'1(x,y)\) represents the brightness component value at the pixel point position \((x,y)\) after brightness enhancement processing, the variable \((p,q)\) represents the pixel points belonging to the edge pixel points, \(V(p,q)\) represents the brightness component value at the pixel point position \((p,q)\), and \(Num(B)\) represents the total number of edge pixel points;

[0083] Perform brightness enhancement processing on the second characteristic region:

[0084]

[0085] Wherein, the variable (x, y) represents the pixel point belonging to the second feature area, V'2(x, y) represents the brightness component value at the pixel point position (x, y) after brightness enhancement processing, the variable (p, q) represents the pixel point belonging to the edge pixel point, V(p, q) represents the brightness component value at the pixel point position (p, q), Num(B) represents the total number of edge pixels; ω1 represents the set weight factor, where 0.4≤ω1≤0.65, V p ((x,y)) represents the brightness component value of the pixel point that is closest to the pixel point position (x,y) and is not in the second feature area;

[0086] The pre-processed accompanying video data is obtained according to the brightness component V' after the brightness enhancement processing.

[0087] The preprocessed accompanying medical consultation video data is further transmitted to the scene extraction module and / or the accompanying medical consultation extraction module.

[0088] In one scenario, the proportional control factors k1=0.1 and k2=0.1 are set;

[0089] In view of the above-mentioned problem that the scene changes and white light sources in the hospital environment may easily cause direct exposure or backlighting in the accompanying video images, the above-mentioned embodiment of the present invention proposes a preprocessing technical solution based on a picture enhancement unit, which can enhance the acquired accompanying video image data and improve the clarity and quality of the image.

[0090] Among them, for the obtained accompanying medical video screen, the edge pixels in the image are first detected based on the edge detection operator, and the video screen is gridded based on the distribution characteristics of the edge pixels (here it is considered that the edge information is easily lost due to the abnormal brightness of the backlight area and the reflective area, and the feature information aggregation area in the image can be obtained according to the edge feature information). An adaptive grid division method is proposed, which can make adaptive judgments according to the number of edge features in the sub-grid, so as to finely divide the part of the image screen where information is concentrated, and for possible lighting abnormalities, it can avoid excessive division, which helps to improve the efficiency and accuracy of subsequent processing.

[0091] Based on the divided sub-grids, the Gaussian eigenvalues ​​of the image are obtained through the HSV color space, and the brightness eigenvalues ​​of each sub-grid are fed back according to the Gaussian eigenvalues. The brightness eigenvalue feedback function proposed can accurately feedback the brightness characteristics of the current sub-grid and accurately distinguish the exposure area and the backlight area. The analysis based on the brightness eigenvalue is based on the horizontal and vertical ratio levels of the brightness eigenvalue, so as to accurately judge the backlight area (first feature area) and exposure area (second feature area) in the image, and perform targeted brightness enhancement adjustment for different areas, thereby improving the overall clarity of the video picture. Among them, for the exposure area and the backlight area, the edge area with the most feature information in the image is used as a reference to adaptively adjust the brightness of the abnormally illuminated area, thereby restoring the abnormal illumination of the abnormal area and improving the clarity and quality of the image.

[0092] By enhancing the acquired medical video data in the above manner, abnormal lighting conditions in the image can be intelligently eliminated, the clarity and quality of the image can be improved, and the foundation for subsequent further video playback and further analysis and processing can be laid.

[0093] Preferably, the scene extraction module specifically includes:

[0094] Performing scene recognition based on the video picture information of the acquired accompanying medical video data, including detecting scene markers in the video picture information, and determining the scene features of the current video picture information based on the detected scene markers;

[0095] The scene markers include signboards, medical equipment, medical staff, etc., and each scene marker corresponds to one or more location information.

[0096] Among them, scene feature extraction can be performed based on the recognition method of scene markers in the image. By pre-marking specific markers in various scenes (such as signs, medical equipment, medical staff, etc.), when the video screen is identified to contain scene markers, the scene corresponding to the marker is extracted as the current scene feature.

[0097] By adding judgments on scene features, the evaluation criteria for accompanying services in each scene can be further refined, thereby further improving the accuracy and pertinence of accompanying service analysis.

[0098] Preferably, the accompanying diagnosis extraction module includes a behavior extraction unit and / or a speech extraction unit; wherein,

[0099] The behavior extraction unit is used to extract behavior features according to the video picture information of the acquired accompanying medical video data, including extracting the hand behavior movements of the current accompanying medical practitioner according to the video picture information, and obtaining corresponding behavior features according to the hand behavior movements, wherein the behavior features include operating a mobile phone, picking up objects, and human contact, etc.;

[0100] The speech extraction unit is used to extract speech features based on the speech information of the acquired accompanying medical consultation video data, including performing speech recognition processing based on the language information, extracting corresponding text information, and obtaining corresponding speech features after cleaning and standardization processing based on the extracted text information, wherein the speech features include the speech text information of the accompanying medical consultation physician and other personnel.

[0101] For behavioral feature detection, the accompanying video screen is used to detect whether the accompanying doctor's hands have abnormal behavior (such as receiving money or playing with mobile phones, etc.). By detecting abnormal behavior, it can avoid the accompanying doctor from violating regulations or improper behavior during the accompanying service, effectively supervise the accompanying service, and improve the quality of the accompanying service;

[0102] For speech feature extraction, the conversation between the accompanying physician and the patient is converted into text information, and based on the text information, the accompanying physician's words or speech are analyzed, so as to analyze the accuracy of the accompanying physician's words during the accompanying service and whether there is any illegal or sensitive language.

[0103] Preferably, the accompanying diagnosis analysis module includes: forming an input vector according to each time feature and the scene feature, behavior feature and speech feature corresponding to the same time, and inputting the input vector into a trained analysis model, and the analysis model performs accompanying diagnosis performance analysis on the corresponding accompanying diagnosis features according to the scene features to obtain accompanying diagnosis performance analysis results.

[0104] Preferably, the accompanying diagnosis analysis module includes a model training unit;

[0105] The model training unit is used to train the analysis model, wherein the analysis model is constructed by using a CNN convolutional neural network based on deep learning technology, wherein the analysis model includes an input layer, a hidden layer and an output layer, wherein the input layer is used to input a constructed input vector, and the hidden layer performs convolution processing on the input data to extract data features, and the output layer sets a Softmax classifier to classify the convolution features to obtain corresponding accompanying medical performance analysis results; wherein in the process of training the analysis model, corresponding environmental features and accompanying medical features are extracted based on pre-prepared standard accompanying medical video data, a training set is formed based on environmental features, accompanying medical features and corresponding calibrated standard performance analysis results, and the analysis model is trained based on the training set until the training model guarantees a certain accuracy rate when testing the test set, thereby obtaining a trained analysis model.

[0106] The analysis model that continues to be built can perform performance analysis based on the behavior and speech characteristics of the accompanying attendants in different scenarios. For different scenarios, it analyzes whether the accompanying attendants accurately express the corresponding guidance words (for example, guiding patients to complete registration or payment, reminding them of relevant precautions, or calming the patients' emotions in time during physical therapy, etc.); for violation detection, the accompanying attendants' speech characteristics are used to analyze whether sensitive words (such as scalpers, fees, and other related sensitive words) appear in the conversation between the accompanying attendants and the patients, so as to analyze whether the accompanying attendants have violated the regulations.

[0107] In the process of building the training set, corresponding training sets can be set according to the above-mentioned normal or abnormal situations, and the training sets can be marked accordingly to improve the construction of the training set; for the general training set, after inputting the corresponding behavioral characteristics or speech characteristics, the corresponding performance scores are further marked (for example, 0 points is the lowest and 5 points is the highest), and the performance scores are used as the basis for the subsequent accompanying performance analysis results.

[0108] Preferably, the intelligent analysis management platform also includes a playback module;

[0109] The playback module is used to play the accompanying video data transmitted back by the wearable acquisition terminal in real time.

[0110] Management personnel can watch the accompanying video data sent back by the accompanying therapist in real time, and promptly understand the accompanying therapist's current service status based on the accompanying video data so as to provide timely guidance or correction.

[0111] Preferably, the wearable data collection terminal is also provided with a call module;

[0112] The call module is used to establish a real-time communication connection with the intelligent analysis management platform;

[0113] The intelligent analysis management platform also includes a second call module;

[0114] The call module is used to establish a real-time call connection with the wearable acquisition terminal.

[0115] Through the call module, managers can have real-time conversations with accompanying service providers and provide them with real-time guidance and prompts.

[0116] Preferably, the intelligent analysis management platform also includes a data management module;

[0117] The data management module is used to perform privacy protection processing on the received accompanying medical consultation video data, and to classify, store and manage the accompanying medical consultation video data after the privacy protection processing, and to build an accompanying medical consultation video database.

[0118] Based on the construction of the accompanying medical consultation video database, the training set of the analysis model can be further improved based on the recorded accompanying medical consultation video data (a new training set is formed after professionals score the historical video data), so as to continuously improve the quality of the system.

[0119] Preferably, the intelligent analysis management platform also includes a medical companion file module;

[0120] The medical companion file module is used to generate a medical companion file based on the medical companion video data and the corresponding medical companion performance analysis results transmitted by the medical companion.

[0121] Recording the service status of each accompanying therapist in the form of archives can help managers or accompanying therapists themselves review the accompanying services and help improve the subsequent service quality.

[0122] It should be noted that each functional unit / module in each embodiment of the present invention may be integrated into one processing unit / module, or each unit / module may exist physically separately, or two or more units / modules may be integrated into one unit / module. The above-mentioned integrated unit / module may be implemented in the form of hardware or in the form of software functional unit / module.

[0123] Through the description of the above implementation mode, it can be clearly understood by those skilled in the art that the embodiments described herein can be implemented in hardware, software, firmware, middleware, code or any appropriate combination thereof. For hardware implementation, the processor can be implemented in one or more of the following units: application specific integrated circuit (ASIC), digital signal processor (DSP), digital signal processing device (DSPD), programmable logic device (PLD), field programmable gate array (FPGA), processor, controller, microcontroller, microprocessor, other electronic units designed to implement the functions described herein or their combination. For software implementation, part or all of the process of the embodiment can be completed by instructing the relevant hardware through a computer program. When implemented, the above program can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein the communication medium includes any medium that facilitates the transmission of a computer program from one place to another. The storage medium can be any available medium that can be accessed by a computer. Computer-readable media can include, but are not limited to, RAM, ROM, EEPROM, CD-ROM or other optical disk storage, disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer.

[0124] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention, rather than to limit the scope of protection of the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should analyze that the technical solution of the present invention can be modified or replaced by equivalents without departing from the essence and scope of the technical solution of the present invention.

Claims

1. A medical companion management system based on artificial intelligence technology, characterized in that: It includes wearable data collection terminal and intelligent analysis and management platform; among them, The wearable data collection terminal is worn on the accompanying physician to collect the accompanying video data recorded during the accompanying process and upload the accompanying video data to the intelligent analysis management platform; The intelligent analysis management platform is used to extract corresponding scene features and accompanying medical features based on the acquired accompanying medical video data, and use the trained analysis model to perform accompanying medical performance analysis based on the acquired scene features and accompanying medical features to obtain accompanying medical performance analysis results, wherein the accompanying medical features include behavioral features and / or speech features.

2. According to the artificial intelligence technology-based accompanying medical management system of claim 1, it is characterized in that: The wearable acquisition terminal includes a video recording module and a wireless transmission module; wherein, The video recording module is used to collect the accompanying video data of the accompanying doctor during the accompanying process, wherein the accompanying video data includes video picture information and voice information; The wireless transmission module is used to transmit the acquired accompanying medical video data to the intelligent analysis and management platform in real time.

3. According to the artificial intelligence technology-based accompanying medical management system of claim 2, it is characterized in that: The intelligent analysis management platform includes a scene extraction module, a medical companion extraction module, a medical companion analysis module and a result output module; among them, The scene extraction module is used to perform scene analysis based on the video picture information of the acquired accompanying video data, and obtain the scene features corresponding to the current accompanying video data, wherein the scene features include the location information of the current accompanying video data, including wards, consulting rooms, treatment rooms, examination rooms, hospital lobbies, pharmacies, registration offices, and cashier offices; The accompanying diagnosis extraction module is used to extract behavior features based on the video picture information of the accompanying diagnosis video data obtained to obtain corresponding behavior features; and / or to extract speech features based on the voice information of the accompanying diagnosis video data obtained to obtain corresponding speech features; and to form accompanying diagnosis features based on the obtained behavior features and / or speech features; The accompanying diagnosis analysis module is used to input the acquired scene features and accompanying diagnosis features into the trained analysis model, and the analysis model performs accompanying diagnosis performance analysis on the corresponding accompanying diagnosis features according to the scene features to obtain accompanying diagnosis performance analysis results; wherein the accompanying diagnosis performance analysis results include accompanying diagnosis performance rating information; The result output module is used to output the corresponding accompanying performance analysis results, and when the accompanying performance analysis results are abnormal, output the corresponding warning message.

4. According to the artificial intelligence technology-based accompanying medical management system of claim 3, it is characterized in that: The intelligent analysis management platform also includes a pre-processing module; The preprocessing module is used to first preprocess the received accompanying medical consultation video data to improve the quality of the accompanying medical consultation video data, and further transmit the preprocessed accompanying medical consultation video data to the scene extraction module and / or the accompanying medical consultation extraction module.

5. According to the artificial intelligence technology-based accompanying medical treatment management system of claim 3, it is characterized in that: The scene extraction module specifically includes: Performing scene recognition based on the video picture information of the acquired accompanying medical video data, including detecting scene markers in the video picture information, and determining the scene features of the current video picture information based on the detected scene markers; The scene markers include signboards, medical equipment, and medical staff, and each scene marker corresponds to one or more location information.

6. According to the artificial intelligence technology-based accompanying medical management system of claim 3, it is characterized in that: The accompanying diagnosis extraction module includes a behavior extraction unit and / or a speech extraction unit; wherein, The behavior extraction unit is used to extract behavior features according to the video picture information of the acquired accompanying medical video data, including extracting the hand behavior movements of the current accompanying medical practitioner according to the video picture information, and obtaining corresponding behavior features according to the hand behavior movements, wherein the behavior features include operating a mobile phone, picking up objects and human contact; The speech extraction unit is used to extract speech features based on the speech information of the acquired accompanying medical consultation video data, including performing speech recognition processing based on the language information, extracting corresponding text information, and obtaining corresponding speech features after cleaning and standardization processing based on the extracted text information, wherein the speech features include the speech text information of the accompanying medical consultation physician and other personnel.

7. According to the artificial intelligence technology-based accompanying medical treatment management system of claim 3, it is characterized in that: The accompanying diagnosis analysis module includes: forming an input vector according to each time feature and the scene features, behavior features and speech features corresponding to the same time, and inputting the input vector into a trained analysis model, and the analysis model performs accompanying diagnosis performance analysis on the corresponding accompanying diagnosis features according to the scene features to obtain the accompanying diagnosis performance analysis results.

8. According to the artificial intelligence technology-based accompanying medical treatment management system of claim 3, it is characterized in that: The accompanying diagnosis analysis module includes a model training unit; The model training unit is used to train the analysis model, wherein the analysis model is constructed by using a CNN convolutional neural network based on deep learning technology, wherein the analysis model includes an input layer, a hidden layer and an output layer, wherein the input layer is used to input a constructed input vector, and the hidden layer performs convolution processing on the input data to extract data features, and the output layer sets a Softmax classifier to classify the convolution features to obtain corresponding accompanying medical performance analysis results; wherein in the process of training the analysis model, corresponding environmental features and accompanying medical features are extracted based on pre-prepared standard accompanying medical video data, a training set is formed based on environmental features, accompanying medical features and corresponding calibrated standard performance analysis results, and the analysis model is trained based on the training set until the training model guarantees a certain accuracy rate when testing the test set, thereby obtaining a trained analysis model.

9. The accompanying medical treatment management system based on artificial intelligence technology according to claim 3 is characterized in that: The intelligent analysis management platform also includes one or more of a playback module, a data management module, a call module, and a medical companion file module; among which, The playback module is used to play the accompanying video data transmitted back by the wearable acquisition terminal in real time; The call module is used to establish a real-time call connection with the wearable acquisition terminal; The data management module is used to perform privacy protection processing on the received accompanying medical video data, and to classify, store and manage the accompanying medical video data after the privacy protection processing, and to build an accompanying medical video database; The medical companion file module is used to generate a medical companion file based on the medical companion video data and the corresponding medical companion performance analysis results transmitted by the medical companion.

Images