Data Processing Method and Equipment for Automated Drinking Water Sensor System in Smart Wards

Abstract

The present application relates to the field of data processing of automatic Kuwata drinking water sensing systems in smart wards, and specifically to data processing methods and devices for automatic Kuwata drinking water sensing systems in smart wards. A specific implementation of the method includes: controlling the Kuwata drinking water sensor to synchronously collect the swallowing vibration information sequence of the target user; generating a vibration frequency spectrum based on the swallowing vibration information sequence; performing swallowing action recognition on the facial image set to generate a Kuwata drinking water swallowing action information set; controlling the medical system to comprehensively perform Kuwata drinking water swallowing disorder rating detection on the first Kuwata drinking water swallowing detection information, the second Kuwata drinking water swallowing detection information and the third Kuwata drinking water swallowing detection information, and generating corresponding swallowing feedback information; synchronizing the generated Kuwata drinking water swallowing disorder rating detection results and swallowing feedback information to the nursing system. This implementation improves the accuracy of the Kuwata drinking water swallowing test.

Description

Technical Field

[0001] The present application relates to the field of data processing of Kubota drinking water sensing systems, and specifically to methods and equipment for processing data of automatic Kubota drinking water sensing systems in smart wards. Background Art

[0002] The clinical Kubota water test is manually conducted by medical staff, usually using a 50 ml syringe, by intermittently pumping water for the patient to swallow, observing the number of swallowing and choking, and manually rating according to the standardized Kubota water test rating. This leads to cumbersome operation. There is no specific pre-filled Kubota water measuring device for operation, and medical staff need to extract the required test fluid by themselves; manual judgment of Kubota water grading has a certain degree of personal subjectivity, and the number of choking cannot be scientifically and accurately judged. The manual injection of Kubota test water has poor accuracy, and the individual differences of operators are too large, which is prone to errors. There is no specific system and device for judgment, recording, and transmission to the ward nursing system and medical system. At this stage, there is no recommended reference system for patients with abnormal test results. The subjective judgment is large and there is no standardization. Summary of the invention

[0003] The content of this application is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this application is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0004] Some embodiments of the present application propose a data processing method, a computer device and a computer-readable storage medium for an automatic drinking water sensing system in a smart ward to solve the technical problems mentioned in the above background technology section.

[0005] In a first aspect, some embodiments of the present application provide a data processing method for an automatic Kubota drinking water sensing system in a smart ward, which is applied to a Kubota drinking water sensing system in a smart ward. The Kubota drinking water sensing system in the smart ward includes: a Kubota drinking water information collection device, a Kubota drinking water sensor, a nursing system, and a medical system. The method includes: controlling the Kubota drinking water information collection device to collect a facial image set corresponding to a target user at a first time and a three-dimensional facial feature information group corresponding to a second time, and controlling the Kubota drinking water sensor to synchronously collect a swallowing vibration information sequence of the target user, wherein the facial image set and the three-dimensional facial feature information group are information collected by the target user during a Kubota drinking and swallowing action; generating two-dimensional facial feature information corresponding to each facial image in the facial image set to obtain a two-dimensional facial feature information set; generating a swallowing three-dimensional facial feature information group corresponding to the swallowing detection object set at the first time according to the three-dimensional facial feature information group. dimensional feature information group; generate the three-dimensional facial target feature information group at the first time according to the above-mentioned swallowing three-dimensional feature information group, the pre-constructed swallowing query information group and the above-mentioned two-dimensional facial feature information set; generate the first Kutan drinking water swallowing detection information corresponding to the above-mentioned first time according to the above-mentioned three-dimensional facial target feature information group; generate a vibration frequency spectrum according to the above-mentioned swallowing vibration information sequence as the second Kutan drinking water swallowing detection information; perform swallowing action recognition on the above-mentioned facial image set to generate a Kutan drinking water swallowing action information set as the third Kutan drinking water swallowing detection information; control the above-mentioned medical system to perform Kutan drinking water swallowing disorder rating detection on the above-mentioned first Kutan drinking water swallowing detection information, the above-mentioned second Kutan drinking water swallowing detection information and the above-mentioned third Kutan drinking water swallowing detection information, and generate corresponding swallowing feedback information; synchronize the generated Kutan drinking water swallowing disorder rating detection result and the above-mentioned swallowing feedback information to the nursing system.

[0006] In a second aspect, the present application also provides a computer device, comprising a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, the method described in any implementation of the first aspect is implemented.

[0007] In a third aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, wherein when the computer program is executed by a processor, the method described in any implementation of the first aspect is implemented.

[0008] The above-mentioned embodiments of the present application have the following beneficial effects: the nursing system only needs to automatically upload the Kubota test results, and the system pops up a window to remind the abnormalities; the medical system determines the appropriate follow-up treatment for the patient according to the data processing situation, and gives suggestions, diet, and treatment conditions, which are uploaded, and a pop-up window is displayed for patients with abnormal levels, and the choking and coughing vibration map is pushed, which is more intuitive and convenient. At the same time, the accuracy of the Kubota drinking and swallowing test is also improved. First, the Kubota drinking water information collection device is controlled to collect the facial image set corresponding to the target user at the first time and the three-dimensional facial feature information group corresponding to the second time, and the Kubota drinking water sensor is controlled to synchronously collect the swallowing vibration information sequence of the target user, wherein the facial image set and the three-dimensional facial feature information group are the information collected by the target user under the Kubota drinking and swallowing action. Therefore, it is convenient to perform a full range of Kubota drinking and swallowing detection on the user according to the user's facial image and swallowing vibration information. Secondly, the two-dimensional facial feature information corresponding to each facial image in the facial image set is generated to obtain a two-dimensional facial feature information set. According to the three-dimensional facial feature information group, the swallowing three-dimensional feature information group corresponding to the swallowing detection object set at the first time is generated. Then, according to the swallowing three-dimensional feature information group, the pre-constructed swallowing query information group and the two-dimensional facial feature information set, the three-dimensional facial target feature information group at the first time is generated. Thus, the Kuwata drinking water swallowing detection information at the current time can be accurately generated with low computational complexity. Then, according to the three-dimensional facial target feature information group, the first Kuwata drinking water swallowing detection information corresponding to the first time is generated. According to the swallowing vibration information sequence, a vibration frequency spectrum is generated as the second Kuwata drinking water swallowing detection information. Swallowing action recognition is performed on the facial image set to generate a Kuwata drinking water swallowing action information set as the third Kuwata drinking water swallowing detection information. Thus, the Kuwata drinking water swallowing action of the user can be detected from multiple angles to improve the accuracy of the detection. Finally, the medical system is controlled to comprehensively perform Kuwata drinking water swallowing detection information, the second Kuwata drinking water swallowing detection information and the third Kuwata drinking water swallowing detection information for Kuwata drinking water swallowing disorder rating detection, and generate corresponding swallowing feedback information. The generated Kubota drinking and swallowing disorder rating test results and the above swallowing feedback information are synchronized to the nursing system. Therefore, the nursing system only needs to automatically upload the Kubota test results, and the system pops up a window to remind abnormalities; the medical system determines the appropriate follow-up treatment for the patient based on the data processing situation, and gives suggestions, diet, and treatment conditions, which are uploaded, and a pop-up window is displayed for patients with abnormal levels, and the choking and coughing vibration map is pushed, which is more intuitive and convenient. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The above and other features, advantages and aspects of the embodiments of the present application will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar figure labels represent the same or similar elements. It should be understood that the drawings are schematic and that the components and elements are not necessarily drawn to scale.

[0010] Figure 1 It is a flow chart of some embodiments of the data processing method of the automatic Kubota drinking water sensing system of the smart ward according to the present application;

[0011] Figure 2 It is a schematic diagram of the structure of a computer device suitable for implementing some embodiments of the present application. DETAILED DESCRIPTION

[0012] Embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present application. It should be understood that the drawings and embodiments of the present application are only for exemplary purposes and are not intended to limit the scope of protection of the present application.

[0013] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other.

[0014] It should be noted that the concepts such as "first" and "second" mentioned in this application are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0015] It should be noted that the modifications of "one" and "plurality" mentioned in the present application are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0016] The names of the messages or information exchanged between multiple devices in the embodiments of the present application are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0017] The present application will be described in detail below with reference to the accompanying drawings and in combination with embodiments.

[0018] Figure 1The flowchart of some embodiments of the data processing method of the automatic Kuwata drinking water sensing system of the smart ward according to the present application is shown. The process 100 of some embodiments of the data processing method of the automatic Kuwata drinking water sensing system of the smart ward according to the present application is shown. The data processing method of the automatic Kuwata drinking water sensing system of the smart ward is applied to the Kuwata drinking water sensing system of the smart ward. The Kuwata drinking water sensing system of the smart ward includes: a Kuwata drinking water information collection device, a Kuwata drinking water sensor, a nursing system, and a medical system, including the following steps:

[0019] Step 101, controlling the Kubota drinking water information collection device to collect a facial image set corresponding to a target user at a first time and a three-dimensional facial feature information group corresponding to a second time, and controlling the Kubota drinking water sensor to synchronously collect a swallowing vibration information sequence of the target user.

[0020] In some embodiments, the execution subject (for example, a computer device) of the data processing method of the automatic Kubota drinking water sensing system in the smart ward can control the Kubota drinking water information collection device to collect the facial image set corresponding to the target user at the first time and the three-dimensional facial feature information group corresponding to the second time, and control the Kubota drinking water sensor to synchronously collect the swallowing vibration information sequence of the target user. Among them, the facial image set and the three-dimensional facial feature information group are information collected by the target user under the Kubota drinking water swallowing action. The Kubota drinking water information collection device can be a camera collection device with 2D and 3D image collection functions, which can collect the facial image of the target user when drinking Kubota. The Kubota drinking water sensor can refer to a vibration sensor, which is used to be attached near the neck of the target user to detect the swallowing vibration information of the user when drinking Kubota. The nursing system can refer to a system controlled by medical staff, which is used to prompt medical staff to take care of the target user. The medical system can refer to a system that diagnoses the patient's pathological information and gives nursing / treatment suggestions. The Kubota drinking water information collection device, the Kubota drinking water sensor, the nursing system, and the medical system are connected to each other in communication. The computer device may refer to a computer processor of the Kubota drinking water sensing system of the smart ward. The target user may refer to a patient who needs to undergo Kubota drinking water swallowing detection.

[0021] The second time is a frame time that has a historical continuous relationship with the first time. The first time may be a time period for the current Kubota drinking and swallowing detection. The facial image set is each facial image of the target user during the Kubota drinking and swallowing period captured by the Kubota drinking and swallowing information collection device. The second time is the previous frame time under the first frame time. For example, if the first time is the current Kubota drinking and swallowing period, then the second time is the previous Kubota drinking and swallowing period. The three-dimensional facial feature information group may represent the facial detection content information of the target user during the Kubota drinking and swallowing period under the second time. The three-dimensional facial feature information may be the corresponding three-dimensional (3D) facial feature information of the target user at the second time. The three-dimensional facial feature information may be information in the form of a vector. That is, the three-dimensional facial feature information may be information representing the corresponding detection features of the detected user's face in the corresponding three-dimensional camera coordinate system at the second time. Each facial feature at the second time has a corresponding detection confidence, that is, each three-dimensional facial feature information has a corresponding confidence, representing the accuracy of the three-dimensional facial feature information representing the corresponding detection features of the detected user's face in the corresponding three-dimensional camera coordinate system. The detection confidence can be a value between 0 and 1. The higher the value, the higher the accuracy of the corresponding three-dimensional facial feature information. The swallowing vibration information sequence can refer to the vibration information of the target user when swallowing Kubota drinking water, which is synchronously collected by the Kubota drinking water sensor corresponding to the first time. For example, one swallowing vibration information corresponds to one time granularity. One time granularity can be 10 seconds.

[0022] Step 102, generating two-dimensional facial feature information corresponding to each facial image in the facial image set to obtain a two-dimensional facial feature information set.

[0023] In some embodiments, the execution subject may generate two-dimensional facial feature information corresponding to each facial image in the facial image set to obtain a two-dimensional facial feature information set. The two-dimensional facial feature information may represent feature information in a two-dimensional (2D) space corresponding to the facial image. The two-dimensional feature information may be information in vector form that represents the image semantics corresponding to the captured facial image.

[0024] In an actual application scenario, the above-mentioned execution subject can generate the two-dimensional facial feature information corresponding to each facial image in the above-mentioned facial image set through the following steps:

[0025] The first step is to perform image preprocessing on the facial image to generate a preprocessed facial image, wherein the image preprocessing may include: format conversion, size scaling, and image normalization.

[0026] In the second step, the pre-processed facial image is input into a pre-trained two-dimensional facial feature extraction model to obtain two-dimensional facial feature information. The two-dimensional image feature extraction model may be a neural network model for extracting image feature information from facial images. In practice, the two-dimensional image feature extraction model may be a multi-layer series of convolutional layers. The two-dimensional facial feature information may represent the semantic content of image features in the pre-processed facial image.

[0027] Step 103: Generate a swallowing three-dimensional feature information group corresponding to the swallowing detection object set at the first time according to the three-dimensional facial feature information group.

[0028] In some embodiments, the execution subject may generate a swallowing three-dimensional feature information group corresponding to the swallowing detection object set at the first time according to the three-dimensional facial feature information group. The swallowing detection object set may be a pre-set facial area of ​​the patient that needs to be detected when swallowing water. For example, the swallowing detection object set may include: the mouth area, the throat area. The swallowing three-dimensional feature information group may represent the feature information of each swallowing detection object in the swallowing detection object set corresponding to the three-dimensional camera coordinate system at the first time. The swallowing three-dimensional feature information may be feature information in the form of a vector.

[0029] In an actual application scenario, the execution subject may generate the swallowing three-dimensional feature information group corresponding to the swallowing detection object set at the first time through the following steps:

[0030] The first step is to select the three-dimensional facial feature information whose corresponding confidence is greater than the preset confidence from the above three-dimensional facial feature information group as candidate three-dimensional facial feature information, and obtain the candidate three-dimensional facial feature information group. Among them, the confidence can represent the detection accuracy of the detection result corresponding to the three-dimensional facial feature information. The higher the confidence, the higher the detection accuracy of the detection result corresponding to the corresponding three-dimensional facial feature information. There is a corresponding confidence for each three-dimensional facial feature information. The three-dimensional facial feature information can represent the characteristic semantic content of the face in the three-dimensional dimension.

[0031] The second step is to obtain the spatial transformation information corresponding to the first time and the second time. The spatial transformation information may be the spatial transformation information between the space where the target user is located at the second time and the space where the target user is located at the first time. In practice, the spatial transformation information may be generated based on the coordinate system transformation between the camera coordinate system corresponding to the first time and the camera coordinate system corresponding to the second time. The spatial transformation information may be transformation information in the form of a vector group, which may characterize the spatial changes of each swallowing detection object between the first time and the second time.

[0032] In the third step, according to the spatial transformation information, feature information transformation is performed on each candidate 3D facial feature information in the candidate 3D facial feature information group to generate swallowing 3D feature information in the target camera coordinate system, and obtain the swallowing 3D feature information group. The target camera coordinate system is the camera coordinate system corresponding to the first time. As an example, the execution subject can perform feature information mapping transformation on each candidate 3D facial feature information in the candidate 3D facial feature information group according to the spatial transformation information.

[0033] Step 104: Generate the three-dimensional facial target feature information group at the first time according to the three-dimensional swallowing feature information group, the pre-constructed swallowing query information group and the two-dimensional facial feature information set.

[0034] In some embodiments, the execution subject may generate the three-dimensional facial target feature information group at the first time according to the swallowing three-dimensional feature information group, the pre-constructed swallowing query information group and the two-dimensional facial feature information set. One swallowing query information corresponds to one swallowing detection object. The object query information may be information in vector form that characterizes the feature information of the swallowing detection object. The swallowing detection object feature information may characterize feature information of various relevant features of the swallowing detection object. For example, the various relevant features may include: swallowing detection object position features, swallowing detection object posture features.

[0035] In an actual application scenario, the execution subject may generate the three-dimensional facial target feature information group at the first time by the following steps:

[0036] The first step is to combine the swallowing three-dimensional feature information group and the swallowing query information group to generate a combined feature information group.

[0037] In the second step, the above-mentioned combined feature information group is input into the first attention mechanism model to obtain attention mechanism information. The first attention mechanism model can be a pre-trained neural network model based on the attention mechanism. For example, the first attention mechanism model can be a Transformer model. The attention mechanism information can be information in vector form.

[0038] The third step is to transform the above two-dimensional facial feature information set into three-dimensional space to generate three-dimensional facial feature information. The above two-dimensional facial feature information set can be transformed into three-dimensional space using a pre-acquired three-dimensional feature conversion model to generate three-dimensional facial feature information.

[0039] In the fourth step, the above-mentioned facial three-dimensional feature information and the above-mentioned attention mechanism information are input into the second attention mechanism model to obtain a three-dimensional facial target feature information group. The second attention mechanism model can be a pre-trained neural network model based on the attention mechanism. For example, the second attention mechanism model can be a Transformer model.

[0040] Step 105: generating the first water drinking and swallowing detection information corresponding to the first time according to the three-dimensional facial target feature information group.

[0041] In some embodiments, the execution subject may generate the first water drinking and swallowing detection information corresponding to the first time according to the three-dimensional facial target feature information group.

[0042] For example, the execution subject may input the three-dimensional facial target feature information group into a pre-trained Kubota drinking water swallowing detection model to generate corresponding Kubota drinking water swallowing detection information as the first Kubota drinking water swallowing detection information. The Kubota drinking water swallowing detection model may be a neural network model for generating Kubota drinking water swallowing detection information. For example, the Kubota drinking water swallowing detection model may be a YOLO model. The Kubota drinking water swallowing detection model may include: a regression model and a classification model. The classification model may output a model of the Kubota drinking water swallowing detection category. The regression model may be a model that outputs confidence. The first Kubota drinking water swallowing detection information may represent the difficulty level category of the target user's swallowing.

[0043] Step 106, generating a vibration frequency spectrum according to the swallowing vibration information sequence as the second Watian drinking water swallowing detection information.

[0044] In some embodiments, the execution subject can generate a vibration frequency spectrum according to the swallowing vibration information sequence as the second Kuda drinking water swallowing detection information. A pivot table function in Excel can be used to generate a pivot trend chart corresponding to the swallowing vibration information sequence as the vibration frequency spectrum.

[0045] In an actual application scenario, the above execution entity can generate a vibration frequency spectrum through the following steps:

[0046] The first step is to set the update configuration information corresponding to the target user table data. Among them, the above-mentioned update configuration information includes: information to be updated, update variable information and update mode information, and the above-mentioned update variable information corresponds to at least one update variable. The information to be updated can be configuration information for locating the position or data in the target user table. For example, the information to be updated can include row and column positioning conditions. The row and column positioning conditions can be used to locate the cells that need to be updated. As an example, the row and column positioning conditions can include: condition: col16=='automatic calculation' && col10=='unit vibration times' && col11=='unit swallowing times'. The above-mentioned condition can be a logical operation expression. For example, the logical operation expression can be an MVEL expression. Through the configured resultCol, the result of condition positioning can be further screened to determine the row and column identifier or cell data to be updated. ResultCol can also be defined with a field value. The above-mentioned update variable information can be configuration information for locating the data in the table corresponding to the variables participating in the update. The above-mentioned update variable information can correspond to at least one update variable. The above-mentioned update variable information can include variable row and column positioning conditions corresponding to each update variable. For example, the variable row and column positioning condition may include: variables:order:condition:col6==${col6}&&col7==${col7}&&col8==${col8}&&col9==${col9}&&col10=='Number of vibrations'&&col11=='Number of swallows'. The variable row and column positioning condition may correspond to two variables participating in the update, namely the order variable and the income variable. The order variable and the income variable respectively correspond to the logical operation expression condition for locating the variable data. Through the variable row and column positioning condition, the table contents of multiple rows and columns corresponding to the order variable and the income variable can be located from the target table data respectively. The above-mentioned update method information may be configuration information for the update method. The above-mentioned update method information corresponds to the update variable in the above-mentioned at least one update variable, and it can be understood that the update method information may be constructed based on the update variable. For example, the update method information may be an arithmetic expression constructed using the variables participating in the update. As an example, the update method information may include: expression:order*income.

[0047] The second step is to read the swallowing vibration data to be updated corresponding to the information to be updated from the target user table data. The cell data can be read from the target user table data according to the row and column positioning conditions to obtain the swallowing vibration data to be updated corresponding to the information to be updated.

[0048] The third step is to add the swallowing vibration data to be updated to the swallowing vibration data table to be updated to obtain an updated swallowing vibration data table. The swallowing vibration data table to be updated may be a list for storing the located swallowing vibration data to be updated.

[0049] The fourth step is to add the above swallowing vibration information sequence to the updated vibration data table.

[0050] The fifth step is to update the target user table data based on the updated swallowing vibration data table, the updated vibration data table and the above-mentioned update method information to obtain the updated target user table data. For example, first, the above-mentioned execution subject can input the above-mentioned update method information into a pre-packaged update method to obtain the table update data. The above-mentioned update method can be a method that takes the update method information as a method input parameter and takes the determined table update data as an output to execute the update method corresponding to the update method information. The above-mentioned update method can directly read the updated vibration data from the above-mentioned updated vibration data table. For example, the update method can be expressed as CalRuleHandler.cal(). Then, the above-mentioned execution subject can update the table update data to the cell position corresponding to the updated swallowing vibration data stored in the above-mentioned updated swallowing vibration data table in the target user table data to update the above-mentioned target user table data.

[0051] Step 6: Generate a data pivot trend chart corresponding to the updated target user table data as a vibration frequency spectrum. The data pivot trend chart corresponding to the updated target user table data can be generated as a vibration frequency spectrum through the pivot table function in Excel.

[0052] Therefore, because the update mode information directly corresponds to the update variable, the swallowing vibration data update table that needs to be updated can be updated according to the swallowing vibration data table stored in the update mode information after the update content is determined according to the update mode information. Therefore, the process of updating the table can be decoupled into the process of update configuration, update mode configuration, and data location configuration, which improves the flexibility of table update.

[0053] Step 107 , performing swallowing action recognition on the facial image set to generate a Kuta drinking water swallowing action information set as the third Kuta drinking water swallowing detection information.

[0054] In some embodiments, the execution subject may perform swallowing action recognition on the facial image set to generate a Kuwata drinking water swallowing action information set as the third Kuwata drinking water swallowing detection information.

[0055] In an actual application scenario, the execution subject can perform swallowing action recognition on the facial image set through the following steps:

[0056] The first step is to extract features from the above facial image set to generate a facial image feature sequence. The facial image set can be extracted using a CNN (Convolutional Neural Networks) convolutional neural network model to generate a facial image feature sequence.

[0057] In the second step, a Kubota drinking and swallowing action information set is generated based on the facial image feature sequence and the pre-trained Kubota drinking and swallowing action recognition model. For example, the facial image feature sequence can be input into the pre-trained Kubota drinking and swallowing action recognition model to obtain the Kubota drinking and swallowing action information set. One facial image feature corresponds to one Kubota drinking and swallowing action information. The Kubota drinking and swallowing action information can represent the Kubota drinking and swallowing action type of the current target user. The Kubota drinking and swallowing action type can include: a single complete Kubota drinking and swallowing action type, a multiple complete Kubota drinking and swallowing action type, and a Kubota drinking and swallowing failure action type. The Kubota drinking and swallowing action recognition model can include: a tongue muscle action recognition classifier, a masticatory muscle action recognition classifier, a swallowing action recognition classifier, and a throat action recognition classifier. The Kubota drinking and swallowing action recognition model can also include: a feature depth extraction model and a key point extraction model. The feature depth extraction model adopts a residual neural network model. The key point extraction model adopts a MobileNet model. Among them, the facial video feature depth extraction model and the key point extraction model are connected in series.

[0058] The above-mentioned tongue muscle action recognition classifier also includes: a first classifier and a first action matching degree determination model. The above-mentioned masticatory muscle action recognition classifier also includes: a second classifier and a second action matching degree determination model. The above-mentioned swallowing action recognition classifier also includes: a third classifier and a third action matching degree determination model. The above-mentioned throat action recognition classifier also includes: a fourth classifier and a fourth action matching degree determination model. The first classifier is used to classify the specific action types corresponding to the tongue muscle action, such as the tongue against the upper palate action, the tongue against the lower jaw action, the tongue forward and backward extension action, etc. The second classifier is used to classify the specific action types corresponding to the masticatory muscle action, such as the cheek puffing action, the tooth biting action, the tooth showing action, etc. The third classifier is used to classify the specific action types corresponding to the swallowing action, such as the throat movement action. The fourth classifier is used to classify the specific action types corresponding to the throat action, such as the throat upper movement action, the throat lower movement action, etc. The first action matching degree determination model, the second action matching degree determination model, the third action matching degree determination model, and the fourth action matching degree determination model respectively pre-store standard actions corresponding to different action types, and the specific standard actions are stored in the form of feature point sets. After the classifier determines the specific action type, the action matching degree determination model can extract the feature points corresponding to the standard action matching the specific action type, combine them, and calculate the similarity as the corresponding action matching degree.

[0059] The above-mentioned Kubota drinking and swallowing action recognition model can be trained by the following steps:

[0060] First, obtain a facial image feature training sample set. The facial image feature training sample includes: sample facial image features and sample labels. One facial image feature training sample corresponds to one classifier type.

[0061] Second, determine the initial Kubota drinking and swallowing action recognition cascade classifier. The initial Kubota drinking and swallowing action recognition cascade classifier includes: initial tongue muscle action recognition classifier, initial masticatory muscle action recognition classifier, initial swallowing action recognition classifier and initial throat action recognition classifier.

[0062] Third, for each initial classifier in the initial Kubota drinking and swallowing action recognition cascade classifier, perform the following training steps:

[0063] 1. Select target facial image feature training samples corresponding to the above initial classifier from the above facial image feature training sample set.

[0064] 2. Input the sample facial image features included in the target facial image feature training sample into the initial classifier to obtain an initial classification recognition result.

[0065] 3. Determine the recognition loss value between the above initial classification recognition result and the corresponding sample label. The recognition loss value between the above initial classification recognition result and the corresponding sample label can be determined by a preset loss function. For example, the preset loss function can be a hinge loss function or a cross entropy loss function.

[0066] 4. In response to determining that the recognition loss value is less than or equal to the preset loss value, the initial classifier is determined as a trained recognition classifier.

[0067] Fourth, each recognition classifier is integrated into a Kubota drinking and swallowing action recognition model.

[0068] Step 108, controlling the medical system to comprehensively perform a rating test of the first, second and third Wadaian water swallowing detection information on the Wadaian water drinking and swallowing detection information, and generating corresponding swallowing feedback information.

[0069] In some embodiments, the execution subject may control the medical system to comprehensively perform Kuwata drinking water swallowing disorder rating detection on the first Kuwata drinking water swallowing detection information, the second Kuwata drinking water swallowing detection information and the third Kuwata drinking water swallowing detection information, and generate corresponding swallowing feedback information. The medical system is provided with a Kuwata drinking water swallowing disorder rating mapping table corresponding to the first Kuwata drinking water swallowing detection information, the second Kuwata drinking water swallowing detection information and the third Kuwata drinking water swallowing detection information, and the Kuwata drinking water swallowing disorder rating mapping table may be used to find the Kuwata drinking water swallowing disorder rating information corresponding to the first Kuwata drinking water swallowing detection information, the second Kuwata drinking water swallowing detection information and the third Kuwata drinking water swallowing detection information. Among them, the Kuwata drinking water swallowing disorder rating information may indicate the Kuwata drinking water swallowing disorder level. The higher the Kuwata drinking water swallowing disorder level, the more serious the Kuwata drinking water swallowing disorder. The swallowing feedback information may refer to the nursing advice / treatment information corresponding to the Kuwata drinking water swallowing disorder level. The swallowing feedback information may be pre-set information corresponding to the level of Kubota's swallowing difficulty.

[0070] Step 109 , synchronizing the generated Kubota drinking water swallowing disorder rating test result and the above swallowing feedback information to the nursing system.

[0071] In some embodiments, the execution subject may synchronize the generated Kubota drinking water swallowing disorder rating test results and the swallowing feedback information to the nursing system, so that medical staff can provide care for the target user based on the Kubota drinking water swallowing disorder rating test results and the swallowing feedback information.

[0072] The present application also provides a computer device 200. Figure 2As shown, the computer device 200 includes: a bus 201, a processor 202, a memory 203 and a communication interface 204. The processor 202, the memory 203 and the communication interface 204 communicate with each other through the bus 201. The computer device 200 can be a server or a terminal device. It should be understood that the present application does not limit the number of processors and memories in the computer device 200.

[0073] The bus 201 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 2 The bus 201 is represented by only one line, but it does not mean that there is only one bus or one type of bus. The bus 201 may include a path for transmitting information between various components of the computer device 200 (for example, the memory 203, the processor 202, and the communication interface 204).

[0074] The processor 202 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0075] The memory 203 may include a volatile memory, such as a random access memory (RAM). The memory 203 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0076] The memory 203 stores executable program codes, and the processor 202 executes the executable program codes to respectively implement the functions of the aforementioned acquisition module, sampling module, determination module, and mixing module, thereby implementing the aforementioned method for processing data of the smart ward automatic field drinking water sensing system. That is, the memory 203 stores instructions for executing the aforementioned method for processing data of the smart ward automatic field drinking water sensing system.

[0077] The communication interface 204 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computer device 200 and other devices or a communication network.

[0078] An embodiment of the present application also provides a chip, which includes a processor and a data interface. The processor reads instructions stored in a memory through the data interface to execute the data processing method of the above-mentioned smart ward automatic drinking water sensing system.

[0079] The embodiment of the present application also provides a computer-readable storage medium. The above-mentioned computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center containing one or more available media. The above-mentioned available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid-state hard disk). The computer-readable storage medium includes instructions, and the above-mentioned instructions instruct the computing device to execute the method for data processing of the above-mentioned smart ward automatic Kubota drinking water sensing system.

[0080] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0081] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the protection scope of the technical solutions of the embodiments of the present application.

Claims

1. A data processing method for an automatic potable water drinking sensing system in a smart ward, characterized in that: The method is applied to the smart ward Kubota drinking water sensing system, the smart ward Kubota drinking water sensing system comprises: a Kubota drinking water information collection device, a Kubota drinking water sensor, a nursing system, and a medical system. The method comprises: Control the Kubota drinking water information collection device to collect a facial image set corresponding to the target user at a first time and a three-dimensional facial feature information group corresponding to a second time, and control the Kubota drinking water sensor to synchronously collect a swallowing vibration information sequence of the target user, wherein the facial image set and the three-dimensional facial feature information group are information collected by the target user during the Kubota drinking water swallowing action; Generate two-dimensional facial feature information corresponding to each facial image in the facial image set to obtain a two-dimensional facial feature information set; generating a swallowing three-dimensional feature information group corresponding to the swallowing detection object set at the first time according to the three-dimensional facial feature information group; generating a three-dimensional facial target feature information group at the first time according to the swallowing three-dimensional feature information group, the pre-constructed swallowing query information group and the two-dimensional facial feature information set; generating first Kuda drinking and swallowing detection information corresponding to the first time according to the three-dimensional facial target feature information group; According to the swallowing vibration information sequence, a vibration frequency spectrum is generated as the second Kuda drinking and swallowing detection information; Perform swallowing action recognition on the facial image set to generate a Kubota drinking water swallowing action information set as third Kubota drinking water swallowing detection information; Controlling the medical system to comprehensively perform Kuwata water drinking and swallowing disorder rating detection on the first Kuwata water drinking and swallowing detection information, the second Kuwata water drinking and swallowing detection information, and the third Kuwata water drinking and swallowing detection information, and generating corresponding swallowing feedback information; Synchronizing the generated Kubota drinking water swallowing disorder rating test results and the swallowing feedback information into the nursing system; The step of performing swallowing action recognition on the facial image set to generate a Kubota drinking water swallowing action information set includes: Extracting features from the facial image set to generate a facial image feature sequence; According to the facial image feature sequence and the pre-trained Kubota drinking and swallowing action recognition model, a Kubota drinking and swallowing action information set is generated; The step of generating the three-dimensional facial target feature information group at the first time according to the swallowing three-dimensional feature information group, the pre-constructed swallowing query information group and the two-dimensional facial feature information set includes: Combining the swallowing three-dimensional feature information group and the swallowing query information group to generate a combined feature information group; Inputting the combined feature information group into a first attention mechanism model to obtain attention mechanism information; Converting the two-dimensional facial feature information set into a three-dimensional space to generate three-dimensional facial feature information; Inputting the facial three-dimensional feature information and the attention mechanism information into a second attention mechanism model to obtain a three-dimensional facial target feature information group; The step of generating the first Kuda drinking and swallowing detection information corresponding to the first time according to the three-dimensional facial target feature information group includes: The three-dimensional facial target feature information group is input into a pre-trained Kuwata water drinking and swallowing detection model to generate corresponding Kuwata water drinking and swallowing detection information as first Kuwata water drinking and swallowing detection information.

2. The data processing method of the smart ward automatic potable water sensing system according to claim 1 is characterized in that: The generating of two-dimensional facial feature information corresponding to each facial image in the facial image set includes: Performing image preprocessing on the facial image to generate a preprocessed facial image; The preprocessed facial image is input into a pre-trained two-dimensional facial feature extraction model to obtain two-dimensional facial feature information.

3. A computer device, wherein: The computer device comprises a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 2 are implemented.

4. A computer-readable storage medium, wherein: The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 2 are implemented.

Images