Food viscosity grading judgment method, system and equipment based on machine learning
Through the food viscosity grading judgment method based on machine learning, a grading judgment model is constructed using video data, which solves the complexity and inaccuracy of existing evaluation methods, and achieves fast and accurate viscosity grading judgment, improving the quality of life of patients.
Patent Information
- Application Number
- CN202411765798.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-04
- Publication Date
- 2025-03-25
AI Technical Summary
Existing methods for food viscosity evaluation, such as syringe experiments and spoon roll experiments, have problems such as operational complexity, lack of unified standards and objective quantitative indicators, and it is difficult to achieve accurate assessment.
Using the food viscosity grading judgment method based on machine learning, a grading judgment model is constructed by receiving and preprocessing food liquidity video data, including video frame image feature extraction, video frame image sequence feature decoding and video prediction classification parts to realize the judgment of IDDSI level.
It achieves rapid and accurate IDDSI level judgment, and improves the quality of life and self-management ability of patients with swallowing dysphagia.
Smart Images
Figure CN119693701A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of food viscosity classification detection, and in particular to a food viscosity classification judgment method, system and equipment based on machine learning. Background Art
[0002] Swallowing dysfunction is a common clinical problem, which refers to the difficulty in swallowing due to physiological or pathological reasons. This disorder may occur at different stages of swallowing, manifesting as difficulty in swallowing, which in turn affects the safe and effective transmission of food from the mouth to the stomach. The main symptoms of swallowing dysfunction include choking when drinking water and difficulty in swallowing, which not only affect the patient's food intake and nutrient absorption, but also may cause food to be accidentally inhaled into the trachea, causing aspiration pneumonia, and in severe cases may even be life-threatening due to suffocation. For people with swallowing disorders, adjusting the texture and consistency of the diet has become a common solution. By increasing the consistency of food, the safety of swallowing can be improved and the risk of airway aspiration can be reduced.
[0003] At present, the internationally recognized standard for assessing the consistency of dysphagia food is the standard developed by the International Dietary Standards Initiative (IDDSI). This standard evaluates the consistency of food through flow testing, in which the determination of the IDDSI level depends on the amount of liquid remaining after 10 seconds. The existing flow testing methods mainly use needle tube test and spoon tilt test, but both methods have certain limitations. Although the needle tube test is regarded as authoritative, its operation complexity and environmental restrictions make it difficult to popularize; and although the spoon tilt test is simple, it is difficult to achieve accurate assessment due to the lack of unified standards and objective quantitative indicators. For this reason, we propose a food viscosity grading judgment method, system and equipment based on machine learning. Summary of the invention
[0004] The purpose of the present invention is to provide a method, system and device for grading and judging food viscosity based on machine learning to solve the problems raised in the above background technology.
[0005] According to a first aspect of the present invention, in order to achieve the above-mentioned purpose, the present invention provides the following technical solution: a method for grading and judging food viscosity based on machine learning, comprising the following steps:
[0006] Receive food fluidity video data, and pre-process the video data to form a data set;
[0007] Divide the preprocessed data set into training set and test set;
[0008] Construct a classification judgment model, input the training set into the classification judgment model for training, and analyze the classification judgment model after each round of training on the test set until the loss function converges to obtain the optimal classification judgment model;
[0009] The video data to be classified and identified is input into the optimal classification judgment model, and the classification judgment result is output.
[0010] Furthermore, the method for collecting food fluidity video data is as follows:
[0011] (1) Sample selection and configuration: Four common thickening products on the market were selected as experimental samples. Each product was configured with five different viscosity levels, corresponding to levels 0 to 4 of the IDDSI standard. Three repeated experiments with different proportions were conducted for each viscosity level.
[0012] (2) Video shooting standards: Videos were shot at 1080p resolution and 60 frames per second. The background was uniformly white. Three videos were shot for each experiment. The indoor light source was kept stable during shooting. A fixed-position fill light and reverse lighting were used. The center of the shot was aimed at the spoon to ensure that the flow of food was clearly visible and the shooting was stable.
[0013] (3) Spoon operation standards:
[0014] (3.1) Food intake standard: simulate the actual usage scenario, put the spoon into the container containing food to take food, and do not limit the amount of food taken;
[0015] (3.2) Pouring standard: The spoon was tilted at an angle of 30, 45, 60 or 90 degrees to simulate the uncertainty of the user's actual operation. The pouring action was kept at a constant speed and the pouring time was 1-2 seconds.
[0016] Furthermore, the preprocessing of the video data specifically includes cropping the collected video, unifying the resolution, performing data enhancement processing, and decomposing the video into continuous video frame images.
[0017] Furthermore, the hierarchical judgment model includes a video frame image feature extraction part, a video frame image sequence feature decoding and a video prediction and classification part. The video frame image feature extraction part replaces the original fully connected layer with an enhancement module to enhance the generalization ability of the model and reduce overfitting while maintaining the original extraction accuracy. The video frame image sequence feature decoding adopts a multi-layer long short-term memory network structure to improve the expression ability of the model and capture longer dependencies. The video prediction and classification part is used to process the video frame image sequence features with time sequence, selects the output of the last time step, processes it through multiple fully connected layers, and finally outputs the prediction result corresponding to the video classification category.
[0018] Furthermore, the enhancement module includes a fully connected layer with an input dimension of 2048 and an output dimension of 1024, a batch normalization layer, a fully connected layer with an input dimension of 1024 and an output dimension of 768, a batch normalization layer, and a fully connected layer with an input dimension of 768 and an output dimension of 512.
[0019] Furthermore, the training set and the test set are divided in a ratio of 8:2.
[0020] Furthermore, the loss function of the hierarchical judgment model is a cross entropy loss function.
[0021] According to a second aspect of the present invention, the present invention provides a video viscosity grading judgment system based on machine learning, comprising:
[0022] A preprocessing module, used for receiving food fluidity video data and preprocessing the video data to form a data set;
[0023] A partitioning module is used to partition the preprocessed data set into a training set and a test set;
[0024] A construction module is used to construct a classification judgment model, input the training set into the classification judgment model for training, and analyze the classification judgment model after each round of training on the test set until the loss function converges to obtain the optimal classification judgment model;
[0025] The output module is used to input the video data to be classified and identified into the optimal classification judgment model and output the classification judgment result.
[0026] According to a third aspect of the present invention, the present invention provides a terminal device, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the memory stores a computer program capable of running on the processor, and when the processor loads and executes the computer program, the above-mentioned food viscosity grading judgment method based on machine learning is adopted.
[0027] According to a fourth aspect of the present invention, the present invention provides a storage medium comprising computer executable instructions, which, when executed by a computer processor, are used to execute the above-mentioned food viscosity grading judgment method based on machine learning.
[0028] The present invention has at least the following beneficial effects:
[0029] The present invention combines a simpler spoon tilting and dropping experiment to judge the IDDSI level. First, video data of the spoon tilting and dropping experiment under different products, different concentrations, and different operating conditions are constructed. Then, the video data is cropped, pixels are adjusted, data amplification and other preprocessing are performed, and a video classification model is trained to realize the unique value of judging the IDDSI level in helping the daily diet management of the families of patients with dysphagia. The invention realizes fast and accurate IDDSI level judgment, thereby improving the patients' quality of life and self-management ability.
[0030] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] Figure 1 It is a schematic diagram of the process of the method of the present invention;
[0032] Figure 2 It is a structural schematic diagram of the hierarchical judgment model in the present invention;
[0033] Figure 3 A schematic diagram of video data acquisition of the present invention;
[0034] Figure 4 This is a schematic diagram of the evaluation results of the grading judgment model in the present invention;
[0035] Figure 5 The figure is a schematic diagram of the operation flow of an embodiment of the present invention. DETAILED DESCRIPTION
[0036] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.
[0037] Embodiment 1:
[0038] See also Figure 1-Figure 4 The present invention provides a technical solution: a method for grading and judging food viscosity based on machine learning, comprising the following steps:
[0039] S1. Receive food fluidity video data and pre-process the video data to form a data set;
[0040] S1.1 Methods for collecting food flowability video data, such as Figure 3 As shown, the details are as follows:
[0041] Use a spoon to place foods of different viscosities on the spoon, then tilt the spoon to observe the flow of the food and shoot a video to ensure that enough video data is collected, including tilts at different angles and speeds, and foods of different viscosities:
[0042] S1.11 Sample selection and configuration: This invention selects 4 common thickening products on the market as experimental samples, and configures 5 different viscosity levels for each product, corresponding to levels 0 to 4 of the IDDSI standard. Three repeated experiments (different ratios) were conducted for each viscosity level, and three videos were shot for each experiment to ensure the diversity and reliability of the data;
[0043] S1.12 Video shooting standard: Use a smartphone to shoot videos at 1080p resolution and 60 frames per second. The background is uniformly white to reduce background interference. The indoor light source is kept stable to avoid changes in natural light. Use a fixed-position fill light and reverse lighting evenly. The smartphone is fixed on a tripod to ensure stable shooting. The center of the shooting is aimed at the spoon to ensure that the flow of food is clearly visible.
[0044] S1.13 Spoon operation standard:
[0045] (1) Eating standards: simulate actual usage scenarios, insert a spoon into a container containing food to eat food, without excessive quantity restrictions, to reflect the diversity of actual operations;
[0046] (2) Pouring standard: The spoon tilting operation is closer to actual use. There is no strict limit on the tilting angle. The shots are taken at angles of approximately 30, 45, 60, and 90 degrees to simulate the uncertainty of the user's actual operation. The pouring action is required to be as uniform as possible and take about 1 second to avoid splashing of food due to pouring too quickly.
[0047] S1.2 Video data preprocessing specifically includes cropping the collected video, unifying the resolution, data enhancement processing, and decomposing it into continuous video frame images;
[0048] S2. Divide the preprocessed data set into a training set and a test set in a ratio of 8:2;
[0049] S3. Construct a hierarchical judgment model, which includes a video frame image feature extraction part, a video frame image sequence feature decoding part, and a video prediction classification part, such as Figure 2 As shown, the details are as follows:
[0050] S31. The video frame image feature extraction part replaces the original fully connected layer with a fully connected layer with an input dimension of 2048 and an output dimension of 1024, a batch normalization layer, a fully connected layer with an input dimension of 1024 and an output dimension of 768, a batch normalization layer, a fully connected layer with an input dimension of 768 and an output dimension of 512, so as to adapt to new tasks while maintaining the original extraction accuracy, enhance the generalization ability of the model and reduce overfitting;
[0051] S32. The feature decoding of video frame image sequence adopts a multi-layer long short-term memory network structure to improve the model's expressiveness and capture longer dependencies, and sets a high-dimensional hidden state to enhance the model's information storage capacity;
[0052] S33. The video prediction classification part is used to process the characteristics of the video frame image sequence with time order, select the output of the last time step, process it through multiple fully connected layers, and finally output the prediction result corresponding to the video classification category;
[0053] S34. Input the training set into the classification judgment model for training, and analyze the classification judgment model after each round of training on the test set until the loss function converges to obtain the optimal classification judgment model;
[0054] The loss function of the hierarchical judgment model is the cross entropy loss function, which is expressed as:
[0055]
[0056] L represents the cross entropy loss value, C is the number of categories, and y i is the i-th element in the one-hot encoding vector of the sample’s true label, p i is the probability that the model predicts that the sample belongs to the i-th category;
[0057] S4. Input the video data to be classified and identified into the optimal classification judgment model, output the classification judgment result, and evaluate the judgment result.
[0058] like Figure 5 As shown, the present invention is further described below in conjunction with specific embodiments:
[0059] 1. Prepare experimental materials: prepare food samples with different viscosities. For example, add thickener to water to prepare 5 samples with different viscosities.
[0060] 2. Perform a spoon tilt experiment: Use a standardized spoon to take an appropriate amount of the prepared sample, and tilt the spoon to a certain angle to allow the sample to slide naturally, while recording the experimental video with a smartphone;
[0061] 3. Video input: input the recorded spoon tilting experiment video into the graded judgment model trained in this embodiment;
[0062] 4. Model prediction: The graded judgment model analyzes the input video and predicts the IDDSI grade of the sample;
[0063] 5. Prediction accuracy: The final evaluation prediction accuracy was 92%.
[0064] In summary, this embodiment combines a simpler spoon tilt drop experiment to judge the IDDSI level. First, the spoon tilt drop experiment video data of different products, different concentrations, and different operating conditions is constructed, and then the video data is cropped, pixels are adjusted, data amplification and other preprocessing are performed, and the video classification model is trained to realize the unique value of IDDSI level judgment in helping the daily diet management of families of patients with dysphagia, and to achieve fast and accurate IDDSI level judgment, thereby improving the quality of life and self-management ability of patients.
[0065] Embodiment 2:
[0066] This embodiment provides a video viscosity grading judgment system based on machine learning, including:
[0067] A preprocessing module, used for receiving food fluidity video data and preprocessing the video data to form a data set;
[0068] A partitioning module is used to partition the preprocessed data set into a training set and a test set;
[0069] A construction module is used to construct a classification judgment model, input the training set into the classification judgment model for training, and analyze the classification judgment model after each round of training on the test set until the loss function converges to obtain the optimal classification judgment model;
[0070] The output module is used to input the video data to be classified and identified into the optimal classification judgment model and output the classification judgment result.
[0071] Specifically, the above-mentioned preprocessing module, division module, construction module and output module can be embedded in a computer processing system. The computer calls the above-mentioned modules to complete the task of grading food viscosity according to the above-mentioned food viscosity grading judgment method based on machine learning; the above-mentioned preprocessing module, division module, construction module and output module can perform operations according to the specific steps given in the above-mentioned food viscosity grading judgment method based on machine learning.
[0072] It should be noted that it should be understood that the division of the various modules of the above system is only the division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated, and these modules can all be implemented in the form of software calling through processing elements; they can also be all implemented in the form of hardware; some modules can also be implemented in the form of software calling through processing elements, and some modules can be implemented in the form of hardware. For example, the preprocessing module can be a separately established processing element, or it can be integrated in a certain chip of the above device. In addition, it can also be stored in the memory of the above device in the form of program code, and called and executed by a certain processing element of the above device. The implementation of other modules is similar. In addition, these modules can be fully or partially integrated together, or they can be implemented independently. The processing element described here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each module above can be completed by the hardware integrated logic circuit in the processor element or the instructions in the form of software.
[0073] For example, the above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application specific integrated circuits (ASIC), or one or more digital singnal processors (DSP), or one or more field programmable gate arrays (FPGA). For another example, when a module is implemented in the form of a processing element scheduling program code, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processors that can call program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).
[0074] Embodiment three:
[0075] The present invention provides a terminal device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor. The memory stores a computer program capable of running on the processor, and when the processor loads and executes the computer program, the above-mentioned food viscosity grading judgment method based on machine learning is adopted.
[0076] It should be noted that the terminal device can be a computer device such as a desktop computer, a laptop computer or a cloud server, and the terminal device includes but is not limited to a processor and a memory. For example, the terminal device can also include input and output devices, network access devices and buses.
[0077] Furthermore, the processor may adopt a central processing unit (CPU). Of course, according to actual usage, other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. may also be adopted. The general-purpose processor may adopt a microprocessor or any conventional processor, etc., and the present application does not impose any restrictions on this.
[0078] Embodiment 4:
[0079] The present invention provides a storage medium containing computer executable instructions, characterized in that the computer executable instructions are used to execute the above-mentioned food viscosity grading judgment method based on machine learning when executed by a computer processor.
[0080] Among them, the computer program can be stored in a computer-readable medium, the computer program includes computer program code, the computer program code can be in the form of source code, object code, executable file or certain middleware, etc. The computer-readable medium includes any entity or device that can carry the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the computer-readable medium includes but is not limited to the above-mentioned components.
[0081] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.
[0082] For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances. When an element is referred to as being "assembled on", "installed on", "fixed on" or "set on" another element, it can be directly on the other element or there can also be a centered element. When an element is considered to be "connected" to another element, it can be directly connected to the other element or there may be a centered element at the same time. The terms "vertical", "horizontal", "up", "down", "left", "right" and similar expressions used herein are only for illustrative purposes and are not intended to be the only implementation method.
[0083] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.
[0084] In the description of this specification, the description with reference to the terms "one embodiment", "example", "specific example", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.
Claims
1. A food viscosity grading method based on machine learning, characterized in that: The following steps are involved: Receive food fluidity video data, and pre-process the video data to form a data set; Divide the preprocessed data set into training set and test set; Construct a classification judgment model, input the training set into the classification judgment model for training, and analyze the classification judgment model after each round of training on the test set until the loss function converges to obtain the optimal classification judgment model; The video data to be classified and identified is input into the optimal classification judgment model, and the classification judgment result is output.
2. The method for grading food viscosity based on machine learning according to claim 1, characterized in that: The method for collecting the food fluidity video data is as follows: (1) Sample selection and configuration: Four common thickening products on the market were selected as experimental samples. Each product was configured with five different viscosity levels, corresponding to levels 0 to 4 of the IDDSI standard. Three repeated experiments with different proportions were conducted for each viscosity level. (2) Video shooting standards: Videos were shot at 1080p resolution and 60 frames per second. The background was uniformly white. Three videos were shot for each experiment. The indoor light source was kept stable during shooting. A fixed-position fill light and reverse lighting were used. The center of the shot was aimed at the spoon to ensure that the flow of food was clearly visible and the shooting was stable. (3) Spoon operation standards: (3.1) Food intake standard: simulate the actual usage scenario, put the spoon into the container containing food to take food, and do not limit the amount of food taken; (3.2) Pouring standard: The spoon was tilted at an angle of 30, 45, 60 or 90 degrees to simulate the uncertainty of the user's actual operation. The pouring action was kept at a constant speed and the pouring time was 1-2 seconds.
3. The method for grading food viscosity based on machine learning according to claim 2, characterized in that: The preprocessing of video data specifically includes cropping the acquired video, unifying the resolution, performing data enhancement processing, and decomposing the video into continuous video frame images.
4. The method for grading and judging food viscosity based on machine learning according to claim 3, characterized in that: The hierarchical judgment model includes a video frame image feature extraction part, a video frame image sequence feature decoding and a video prediction classification part. The video frame image feature extraction part replaces the original fully connected layer with an enhancement module to enhance the generalization ability of the model and reduce overfitting while maintaining the original extraction accuracy. The video frame image sequence feature decoding adopts a multi-layer long short-term memory network structure to improve the model's expression ability and capture longer dependencies. The video prediction classification part is used to process video frame image sequence features with time sequence, selects the output of the last time step, processes it through multiple fully connected layers, and finally outputs the prediction result corresponding to the video classification category.
5. The method for grading and judging food viscosity based on machine learning according to claim 4, characterized in that: The enhancement module includes a fully connected layer with an input dimension of 2048 and an output dimension of 1024, a batch normalization layer, a fully connected layer with an input dimension of 1024 and an output dimension of 768, a batch normalization layer, and a fully connected layer with an input dimension of 768 and an output dimension of 512.
6. The method for grading food viscosity based on machine learning according to claim 5, characterized in that: The training set and the test set are divided in a ratio of 8:
2.
7. The method for grading food viscosity based on machine learning according to claim 5, characterized in that: The loss function of the hierarchical judgment model is the cross entropy loss function, which is expressed as: L represents the cross entropy loss value, C is the number of classification categories, y i is the i-th element in the one-hot encoding vector of the true label of the sample, p i is the probability that the model predicts that the sample belongs to the i-th class.
8. A video viscosity grading judgment system based on machine learning, characterized in that: include: A preprocessing module, used for receiving food fluidity video data and preprocessing the video data to form a data set; A partitioning module is used to partition the preprocessed data set into a training set and a test set; A construction module is used to construct a classification judgment model, input the training set into the classification judgment model for training, and analyze the classification judgment model after each round of training on the test set until the loss function converges to obtain the optimal classification judgment model; The output module is used to input the video data to be classified and identified into the optimal classification judgment model and output the classification judgment result.
9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that: The memory stores a computer program that can be run on the processor. When the processor loads and executes the computer program, the method for grading and judging food viscosity based on machine learning described in any one of claims 1 to 7 is adopted.
10. A storage medium containing computer executable instructions, characterized in that: The computer executable instructions are used to execute the food viscosity grading and judging method based on machine learning as described in any one of claims 1 to 7 when executed by a computer processor.