Intelligent control system for old-age care robot based on Internet of Things

Abstract

The invention relates to the technical field of intelligent control, in particular to an internet-of-things-based old-age care robot intelligent control system, which comprises a multi-mode sensing layer, a safety decision center, an execution driving module, a voice interaction optimization module and an internet-of-things communication module. According to the method, force sense data such as contact force of the mechanical arm, voice data such as voices of old people and view data such as obstacles are collected in real time through a multi-mode sensing layer, a displacement force coupling risk value is calculated through a hyperbolic tangent function coupling model of a safety decision center, and a deviation angle risk value is analyzed and calculated through an improved Lyapunov index spectrum; a total risk value is generated through fusion and is scheduled according to hierarchical intervals, mechanical operation, old people behaviors and environmental risks can be dynamically avoided, and the use safety is remarkably improved; through sub-ambiguity quantitative analysis, comprehensive ambiguity evaluation and intention correction of the voice interaction optimization module, instruction intention is optimized in combination with multi-modal data, the problem of fuzzy voice recognition of old people is effectively solved, and interaction accuracy and response efficiency are greatly improved.

Description

Technical Field

[0001] This invention relates to the field of intelligent control technology, specifically to an intelligent control system for elderly care robots based on the Internet of Things. Background Technology

[0002] As an important intelligent device for addressing population aging, existing elderly care robots can effectively fill the gap in elderly care resources. Through functions such as health monitoring and daily living assistance, they can improve the elderly's ability to live independently, reduce the burden of family and social care, and serve as the core support for intelligent elderly care. Currently, in addition to basic functions such as health monitoring, home control, and emotional companionship, elderly care robots are adding daily living assistance functions to help elderly people with limited mobility to move around, turn over, and deliver and move items. However, when performing daily living assistance functions, the operation of the robotic arm is relatively stiff, which can easily injure the elderly when they resist. Furthermore, when interacting with voice commands, the recognition rate for some ambiguous expressions is low, making them inconvenient to use. Summary of the Invention

[0003] This invention provides an intelligent control system for elderly care robots based on the Internet of Things (IoT) to solve the aforementioned technical problems.

[0004] The first aspect of this invention provides an intelligent control system for elderly care robots based on the Internet of Things (IoT), including a multimodal perception layer, a safety decision-making center, an execution drive module, a voice interaction optimization module, and an IoT communication module.

[0005] The multimodal perception layer collects force perception data, speech perception data, and view perception data through multimodal sensors. It then aggregates these data into a multimodal data packet, which is sent to the safety decision center and the voice interaction module. The force perception data includes the contact force, displacement, joint angle data, and current feedback force for each robotic arm. The speech perception data includes voice command information, behavioral body information, and speech-to-text recognition and speech rate information corresponding to each user's speech. The view perception data includes obstacle data, environmental data on ground flatness, and facial expressions and lip movement features corresponding to each user's speech.

[0006] The security decision-making center includes a force control decision-making unit and a security scheduling unit. The force control decision-making unit is used to receive multi-mode data packets and perform behavioral risk assessment and analysis on the multi-mode data packets to obtain the total risk value, and then transmits the total risk value to the security scheduling unit. The security scheduling unit is used to perform scheduling analysis on the total risk value to obtain the corresponding scheduling instructions, and then sends the scheduling instructions to the execution drive module.

[0007] As a further improvement to the present invention, behavioral risk assessment and analysis are performed on multi-mode data packets, and the specific analysis method is as follows:

[0008] Based on the multi-mode data packet, the current force perception data, voice perception data, and view perception data are obtained; based on the force perception data, the contact force, displacement, and joint angle data of each robotic arm are obtained; the displacement is calculated by subtracting the displacement from the corresponding preset displacement target to obtain the displacement deviation; and the robotic arm contact force and displacement deviation are substituted into the hyperbolic tangent function coupling model. The displacement-force coupling risk value jx was calculated. ND Where F and d represent the contact force and displacement deviation of the robotic arm, respectively; F0 and d0 represent the preset upper limit of body tolerance and the allowable displacement deviation, respectively; tanh(x) represents the hyperbolic tangent function; α N α D All of these are preset weighting coefficients.

[0009] The preset motion execution cycle is obtained based on the mechanical joint angle data, and the motion execution cycle is divided into multiple motion monitoring points based on the preset acquisition interval. The joint angle offset vector corresponding to each motion monitoring point is obtained based on the mechanical joint angle data, and the modified Lyapunov exponential spectrum analysis formula is used. The offset angle risk value jx is calculated. θ ; where λ max (t) represents the Lyapunov large exponent of the angular velocity of the robotic arm joint, characterizing the hybridity of motion; Δq(t) = [Δq1,...,Δq n ] represents the joint angle offset vector corresponding to each motion monitoring point; T represents the motion execution cycle; the mechanical risk value is obtained by calculating and summing the displacement force coupling risk value and the offset angle risk value corresponding to the current moment.

[0010] Based on the current speech perception data and view perception data, speech command information and behavioral body information are obtained. The speech command intent of the elderly person is obtained based on the speech command information, and this intent is matched against a pre-stored risk fuzzy command library in the database. When a speech command intent matches the risk fuzzy command library, the corresponding speech command intent is marked as a risk speech command. The confidence level of the speech command is calculated by comparing the number of risk speech commands in the current analysis period with the total number of corresponding speech command intents. The amount of change in the elderly person's movements is obtained through behavioral body information, and the rate of change in movements is calculated by comparing the amount of change in movements with a preset base unit time. Finally, the formula of a preset posture-speech exponential function fusion model is used. The behavioral risk value xw is calculated; where 1-exp(-x) represents the exponential growth function; α y αdz All represent preset weighting factors; v and v0 represent the rate of change of action and the preset rate of change of normal posture, respectively; C represents the confidence level of speech execution.

[0011] Environmental data is obtained through view-aware data. Obstacle data and ground flatness are then derived from this environmental data. The distance to each obstacle is calculated based on the obstacle data. Furthermore, the ground flatness deviation is calculated by comparing the ground flatness with a preset flatness benchmark. Finally, the distance attenuation correction model is used. The environmental risk value dm is calculated; where d z Δh and Δh represent the obstacle distance and ground flatness deviation, respectively; 1 / d z It is represented as the distance inverse ratio parameter; Δh0 represents the flatness deviation threshold.

[0012] The mechanical risk value, behavioral risk value, and environmental risk value are all normalized and their values ​​are recorded as G. jx G xw and G dm Substitute the normalized values ​​into the formula of the nonlinear fusion model. The total risk value G is calculated; where G i ∈[G jx G xw G dm ], β i This represents the preset weighting factor corresponding to each risk value.

[0013] As a further improvement to the present invention, a scheduling analysis is performed on the total risk value, and the specific analysis method is as follows:

[0014] Obtain the pre-set risk classification value G min G high G max Based on each risk classification value, the risk classification interval [0, G] is obtained. min ]、(G min G high ]、(G high G max ]、(G max ,∞), match the current total risk value with each risk level value, when G∈[0,G min When G∈(G), the risk state is generated as normal operation; when G∈(G) min G high When G∈(G), the risk status is low, and the corresponding response measures are early warning prompts and voice inquiries; high G max When G∈(G), the risk status is medium risk, and the corresponding response measures are downgraded operation and voice inquiry;max When the risk value is ∞, the risk status is high risk and the corresponding response measure is emergency termination; the emergency response measure corresponding to the current total risk value and the risk status is marked as a dispatch instruction.

[0015] The execution driver module is used to acquire the intent of the voice command and drive its execution according to the intent of the voice command. It also receives scheduling instructions, identifies the corresponding specific execution items, and drives the execution according to the specific execution items. At the same time, it starts the voice interaction module to conduct voice interaction with the user, acquire the user's voice information, and transmit the voice information to the security decision center.

[0016] The voice interaction optimization module is used to receive multi-mode data packets, perform voice interaction optimization analysis on the multi-mode data packets to obtain optimization strategies, optimize the command intent based on the optimization strategies to obtain voice command intents, and send the voice command intents to the execution driver module.

[0017] As a further improvement to the present invention, voice interaction optimization analysis is performed on multi-mode data packets, and the specific analysis method is as follows:

[0018] Voice interaction optimization analysis obtains the multi-mode data packets corresponding to the current voice interaction, and the voice interaction analysis includes sub-fuzzy quantification analysis, comprehensive fuzziness evaluation, and intent optimization correction;

[0019] Subfuzzy metric analysis analyzes multimode data packets to obtain multiple subfuzzy metric values ​​corresponding to the current voice interaction;

[0020] The comprehensive fuzziness assessment obtains comprehensive fuzziness assessment information by comprehensively analyzing the quantified values ​​of sub-fuzzy metrics;

[0021] Intent optimization correction generates an intent optimization strategy based on comprehensive ambiguity evaluation information, and the intent optimization is performed through the intent optimization strategy to obtain the intent of the voice command.

[0022] Furthermore, the sub-fuzzy quantitative analysis is performed, and its specific analysis content is as follows:

[0023] Force perception data, speech perception data, and view perception data are obtained based on the multi-modal data packets corresponding to the current voice interaction. Speech-text recognition information and speech rate information corresponding to each user's speech information are obtained based on the speech perception data. The user's current speech rate is obtained based on the speech rate information, and the speech rate deviation is calculated by comparing the user's current speech rate with the average speech rate within a preset time period. Based on the speech-text recognition information, text recognition accuracy and semantic integrity scores are obtained using the multi-dimensional feature weighting method formula: ym = 1 - (ω... z1 ×Y sbd +ω z2 ×(1-Y pld )+ω z3×Y wzd The speech ambiguity ym is calculated; where Y sbd Y pld Y wzd These are respectively represented as text recognition accuracy, speech rate deviation, and semantic integrity score; ω z1 ω z2 ω z3 All are preset feature weight factors; obtain the preset speech ambiguity threshold, and when the speech ambiguity is greater than the speech ambiguity threshold, mark the part of the speech ambiguity exceeding the speech ambiguity threshold as the speech sub-ambiguity quantification value.

[0024] Based on the database, the mean and standard deviation of force feedback corresponding to the user's daily cooperation are obtained. The current feedback force is obtained based on the force perception, using a formula based on a normal distribution-dynamic weight model. The force perception ambiguity lm is calculated; where F lm σ lm μ lm Let ω represent the current feedback force, the standard deviation of the force feedback, and the mean of the force feedback, respectively; lm This represents the preset dynamic weight, whose value is dynamically set based on the number of samples of the force signal; E lm This is represented as the preset force feedback deviation tolerance; the preset upper limit of force ambiguity is obtained; when the force ambiguity is greater than the upper limit of force ambiguity, the corresponding force ambiguity is marked as the force ambiguity sub-ambiguity quantification value.

[0025] Based on view-aware data, facial expression information and lip movement features corresponding to each user's voice information are obtained. The facial expression information and lip movement features are then matched with their corresponding voice information to determine their matching degree. The matching degree is output as the matching degree of facial expression commands and the matching degree of lip movement commands, where the matching degree is the historical set of facial expression information and the historical set of lip movement features corresponding to each voice information. When there is no match with the corresponding set, the matching degree of the corresponding command is zero; when there is a match, the matching degree of the corresponding command is the matching value given by the preset design. The matching degree of facial expression commands and the matching degree of lip movement commands are then applied using the multimodal matching algorithm formula st=bc1×(P bq -bq)+bc2×(P cd -cd) calculates the view blur degree st; where bq and cd represent the facial expression command matching degree and lip movement command matching degree, respectively; P bq P cd These represent the matching values ​​corresponding to the facial expression command matching degree and the lip movement command matching degree, respectively; bc1 and bc2 are both preset weighting factors; the view blur is marked as the view sub-blur quantification value.

[0026] Furthermore, the comprehensive ambiguity assessment is specifically conducted as follows: the ambiguity quantification values ​​of the speech sub-fuzziness quantification value, the force sub-fuzziness quantification value, and the view sub-fuzziness quantification value are all normalized, and their values ​​are denoted as M respectively. ym M lm M st Substitute the quantified values ​​of each sub-fuzzy metric into the formula of the weighted correction-membership transformation method. The overall ambiguity M is calculated; where, All are preset weighting factors; K lm Represented as the set force correction coefficient; K st This represents the set correction factor.

[0027] Furthermore, the intention is to optimize and correct, specifically by: obtaining pre-set fuzzy level classification values, namely M1, M2, and M3, and satisfying M1 < M2 < M3; and matching the current corresponding comprehensive fuzziness with each fuzzy level classification value.

[0028] When M < M1, the fuzziness level is low fuzziness, and the optimization strategy is to directly use the speech recognition results;

[0029] When M1 < M < M2, the fuzziness level is medium fuzziness, and the optimization strategy is to base it on the speech results and supplement it with view data;

[0030] When M2 < M < M3, the fuzziness level is high fuzziness, and the optimization strategy is based on the optimization of medium fuzziness and combined with force perception data for verification.

[0031] When M > M3, the fuzziness level is extremely high, and the optimization strategy is to ignore voice commands and prioritize force and visual data.

[0032] The system linkage module is used to store force perception data, voice perception data, and view perception data, and coordinates the various modules through the Internet of Things.

[0033] The beneficial effects of the technical solution provided by this invention compared with the prior art are as follows:

[0034] 1. This invention collects force data such as the contact force of the robotic arm, voice data such as the elderly person's voice, and visual data such as obstacles in real time through a multimodal perception layer. The displacement force coupling risk value is calculated by the hyperbolic tangent function coupling model of the safety decision center, and the offset angle risk value is calculated by the improved Lyapunov exponential spectrum analysis. The total risk value is generated by fusion and scheduled according to hierarchical intervals. This can dynamically avoid risks from mechanical operation, elderly behavior and environment, and significantly improve the safety of use.

[0035] 2. This invention effectively solves the problem of fuzzy speech recognition for the elderly by using sub-fuzzy quantification analysis, comprehensive fuzziness evaluation and intent correction in the voice interaction optimization module, combined with multimodal data to optimize command intent, and significantly improves interaction accuracy and response efficiency. Attached Figure Description

[0036] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. The following drawings are not deliberately drawn to scale according to the actual size, but are intended to show the main idea of ​​this application.

[0037] Figure 1 This is a schematic diagram of the principle of the present invention;

[0038] Figure 2 This is a schematic diagram of the safety decision-making center of the IoT-based intelligent control system for elderly care robots of the present invention.

[0039] Figure 3 This is a schematic diagram of the voice interaction optimization module of the IoT-based intelligent control system for elderly care robots of the present invention. Detailed Implementation

[0040] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0041] For ease of understanding, the specific process of the embodiments of the present invention is described below. Please refer to [link / reference]. Figure 1-3 In one embodiment of the present invention, the intelligent control system for elderly care robots based on the Internet of Things includes:

[0042] The multimodal perception layer acquires force perception data, voice perception data, and view perception data through multimodal sensors; it then combines these data to form a multimodal data packet, which is then sent to the security decision center and the voice interaction module.

[0043] Force perception data includes the contact force, displacement, joint angle data, and current feedback force of each robotic arm; voice perception data includes voice command information and behavioral body information, as well as the voice-text recognition information and speech rate information corresponding to each voice message of the user; view perception data includes obstacle data and environmental data on ground flatness, as well as the facial expression information and lip movement features corresponding to each voice message of the user.

[0044] The security decision-making center includes a force control decision-making unit and a security scheduling unit. The force control decision-making unit is used to receive multi-mode data packets and perform behavioral risk assessment and analysis on the multi-mode data packets to obtain the total risk value, and then transmits the total risk value to the security scheduling unit. The security scheduling unit is used to perform scheduling analysis on the total risk value to obtain the corresponding scheduling instructions, and then sends the scheduling instructions to the execution drive module.

[0045] The behavioral risk assessment and analysis of multi-mode data packets is conducted using the following specific methods:

[0046] A1: Obtain the current force perception data, voice perception data, and view perception data based on the multi-mode data packet.

[0047] A2: Based on the force perception data, obtain the contact force, displacement, and joint angle data of each robotic arm of the robot.

[0048] A21: The displacement deviation is obtained by calculating the difference between the displacement and the corresponding preset displacement target. The contact force of the robotic arm and the displacement deviation are then substituted into the hyperbolic tangent function coupling model. The displacement-force coupling risk value jx was calculated. ND Where F and d represent the contact force and displacement deviation of the robotic arm, respectively; F0 and d0 represent the preset upper limit of body tolerance and the allowable displacement deviation, respectively; tanh(x) represents the hyperbolic tangent function; α N α D All of these are preset weighting coefficients.

[0049] A23: Based on the mechanical joint angle data, a preset motion execution cycle is obtained, and the motion execution cycle is divided into multiple motion monitoring points based on the preset acquisition interval. The joint angle offset vector corresponding to each motion monitoring point is obtained based on the mechanical joint angle data, and the modified Lyapunov exponential spectrum analysis formula is used. The offset angle risk value jx is calculated. θ ; where λ max (t) represents the Lyapunov large exponent of the angular velocity of the robotic arm joint, characterizing the hybridity of motion; Δq(t) = [Δq1,...,Δq n ] represents the joint angle offset vector corresponding to each motion monitoring point; T represents the motion execution cycle.

[0050] A24: The mechanical risk value is obtained by calculating and summing the displacement force coupling risk value and the offset angle risk value corresponding to the current moment.

[0051] A3: Obtain voice command information and behavioral body information based on the current corresponding voice perception data and view perception data.

[0052] A31: Based on voice command information, the intention of the elderly's voice commands is obtained. The intention of the voice commands is matched with the risk fuzzy command library pre-stored in the database. The risk fuzzy command library consists of explicit or implicit feedback that directly expresses physical discomfort, resistance to operation, pain feedback, or emergency stop request, including but not limited to "lighter," "almost done," "pain in a certain place," or painful groans. When the intention of the voice command matches the risk fuzzy command library, the corresponding intention of the voice command is marked as a risk voice command. The confidence level of the voice command is calculated by calculating the ratio of the number of risk voice commands corresponding to the total number of corresponding voice command intentions in the current analysis period.

[0053] A32: Obtain the amount of change in the elderly’s movements through behavioral and bodily information, and calculate the rate of change of movements by comparing the amount of change with a preset base unit of time.

[0054] A33: Utilizing the formula of a pre-defined gesture-speech exponential function fusion model The behavioral risk value xw is calculated; where 1-exp(-x) represents an exponential growth function, which is flat at low risk and rapidly approaches 100 at high risk; α y α dz All are represented as preset weighting factors; v and v0 represent the rate of change of movement and the preset rate of change of normal posture, respectively; C represents the confidence level of speech execution; combining speech and body information to assess behavioral risk and promptly identify resistance or discomfort in the elderly.

[0055] A4: Environmental data is obtained through view perception data. Obstacle data and ground flatness are then derived from the environmental data. The distance to each obstacle is calculated based on the obstacle data. The ground flatness deviation is calculated by comparing the ground flatness with a preset flatness benchmark. The distance attenuation correction model formula is then used. The environmental risk value dm is calculated; where d z Δh and Δh represent the obstacle distance and ground flatness deviation, respectively; 1 / d z It is represented as an inverse distance parameter, with higher risk the closer the distance; Δh0 represents the flatness deviation threshold.

[0056] A5: Normalize the mechanical risk value, behavioral risk value, and environmental risk value, and record their values ​​as G. jx G xw and G dm Substitute the normalized values ​​into the formula of the nonlinear fusion model. The total risk value G is calculated; where G i ∈[G jx G xw G dm ], β iThis represents the preset weighting factor corresponding to each risk value.

[0057] The total risk value is analyzed using the following specific methods:

[0058] Obtain the pre-set risk classification value G min G high G max Based on each risk classification value, the risk classification interval [0, G] is obtained. min ]、(G min G high ]、(G high G max ]、(G max ,∞), match the current total risk value with each risk level value, when G∈[0,G min When G∈(G), the risk state is generated as normal operation; when G∈(G) min G high When G∈(G), the risk status is low, and the corresponding response measures are early warning prompts and voice inquiries; high G max When G∈(G), the risk status is medium risk, and the corresponding response measures are downgraded operation and voice inquiry; max When the risk value is ∞, the risk status is high risk and the corresponding response measure is emergency termination; the emergency response measure corresponding to the current total risk value and the risk status is marked as a dispatch instruction.

[0059] The execution driver module acquires the intent of the voice command and drives its execution accordingly. It also receives scheduling instructions, identifies the corresponding specific execution items, and drives their execution based on those specific items. Simultaneously, it activates the voice interaction module to interact with the user, acquires the user's voice information, and transmits it to the security decision center.

[0060] The voice interaction optimization module receives multi-mode data packets and performs voice interaction optimization analysis on the multi-mode data packets to obtain optimization strategies. Based on the optimization strategies, it optimizes the command intent to obtain the voice command intent and sends the voice command intent to the execution driver module.

[0061] The specific analysis method for optimizing voice interaction using multi-mode data packets is as follows:

[0062] Voice interaction optimization analysis obtains the multi-mode data packets corresponding to the current voice interaction, and the voice interaction analysis includes sub-fuzzy quantification analysis, comprehensive fuzziness evaluation, and intent optimization correction;

[0063] Subfuzzy metric analysis analyzes multimode data packets to obtain multiple subfuzzy metric values ​​corresponding to the current voice interaction;

[0064] The comprehensive fuzziness assessment obtains comprehensive fuzziness assessment information by comprehensively analyzing the quantified values ​​of sub-fuzzy metrics;

[0065] Intent optimization correction generates an intent optimization strategy based on comprehensive ambiguity evaluation information, and the intent optimization is performed through the intent optimization strategy to obtain the intent of the voice command.

[0066] Subfuzzy quantitative analysis, the specific analysis content of which is as follows:

[0067] Force perception data, speech perception data, and view perception data are obtained based on the multi-modal data packets corresponding to the current voice interaction. Speech-text recognition information and speech rate information corresponding to each user's speech information are obtained based on the speech perception data. The user's current speech rate is obtained based on the speech rate information, and the speech rate deviation is calculated by comparing the user's current speech rate with the average speech rate within a preset time period. Based on the speech-text recognition information, text recognition accuracy and semantic integrity scores are obtained using the multi-dimensional feature weighting method formula: ym = 1 - (ω... z1 ×Y sbd +ω z2 ×(1-Y pld )+ω z3 ×Y wzd The speech ambiguity ym is calculated; where Y sbd Y pld Y wzd These are represented as text recognition accuracy, speech rate deviation, and semantic completeness score, respectively. Text recognition accuracy is the ratio of recognizable text to the overall text; semantic completeness score corresponds to the completeness of the core action word and object word of the instruction, for example, values ​​are 0 and 1, where 0 is the score when key information is missing and 1 is the score when the information is complete; ω z1 ω z2 ω z3 All are preset feature weight factors, with values ​​satisfying ω z1 +ω z2 +ω z3 =2; Obtain the preset speech ambiguity threshold. When the speech ambiguity is greater than the speech ambiguity threshold, mark the part of the speech ambiguity that exceeds the speech ambiguity threshold as the speech sub-ambiguity quantification value.

[0068] Based on the database, the mean and standard deviation of force feedback corresponding to the user's daily cooperation are obtained. The current feedback force is obtained based on the force perception, using a formula based on a normal distribution-dynamic weight model. The force perception ambiguity lm is calculated; where F lm σ lm μ lm Let ω represent the current feedback force, the standard deviation of the force feedback, and the mean of the force feedback, respectively; lmThis represents a preset dynamic weight, the value of which is dynamically set based on the number of force signal samples. For example, the more force signal samples, the greater the dynamic weight; E lm This is represented as the preset force feedback deviation tolerance; the preset upper limit of force ambiguity is obtained; when the force ambiguity is greater than the upper limit of force ambiguity, the corresponding force ambiguity is marked as the force ambiguity sub-ambiguity quantification value.

[0069] Based on view-aware data, facial expression information and lip movement features corresponding to each user's voice information are obtained. The facial expression information and lip movement features are then matched with their corresponding voice information to determine their matching degree. The matching degree is output as the matching degree of facial expression commands and the matching degree of lip movement commands, where the matching degree is the historical set of facial expression information and the historical set of lip movement features corresponding to each voice information. When there is no match with the corresponding set, the matching degree of the corresponding command is zero; when there is a match, the matching degree of the corresponding command is the matching value given by the preset design. The matching degree of facial expression commands and the matching degree of lip movement commands are then applied using the multimodal matching algorithm formula st=bc1×(P bq -bq)+bc2×(P cd -cd) calculates the view blur degree st; where bq and cd represent the facial expression command matching degree and lip movement command matching degree, respectively; P bq P cd These represent the matching values ​​corresponding to the facial expression command matching degree and the lip movement command matching degree, respectively; bc1 and bc2 are both preset weighting factors; the view blur is marked as the view sub-blur quantification value.

[0070] The comprehensive ambiguity assessment is specifically conducted as follows: the ambiguity metrics of the speech sub-sub ... ym M lm M st Substitute the quantified values ​​of each sub-fuzzy metric into the formula of the weighted correction-membership transformation method. The overall ambiguity M is calculated; where, All are preset weighting factors; K lm This is expressed as a set force feedback correction coefficient, for example, K when the force feedback is active coordination. lm =0.8, K when there is no feedback lm =0.5, K when resisting lm =1.2, reinforcing the force perception weight under resistant conditions; K st This is represented by a set correction factor, for example, K when the face is directly facing the camera. st =1.0, K when the face is in profile or blurred st =0.6, weakening the impact of invalid views.

[0071] The intention is to optimize and correct the fuzziness, specifically by: obtaining the pre-set fuzziness level classification values, namely M1, M2, and M3, and satisfying M1 < M2 < M3; and matching the current corresponding comprehensive fuzziness with each fuzziness level classification value.

[0072] When M < M1, the fuzziness level is low fuzziness, and the optimization strategy is to directly use the speech recognition results;

[0073] When M1 < M < M2, the fuzziness level is medium fuzziness, and the optimization strategy is to base it on the speech results and supplement it with view data;

[0074] When M2 < M < M3, the fuzziness level is high fuzziness, and the optimization strategy is based on the optimization of medium fuzziness and combined with force perception data for verification.

[0075] When M > M3, the fuzziness level is extremely high, and the optimization strategy is to ignore voice commands and prioritize force and visual data.

[0076] The system's interconnected modules store force perception data, voice perception data, and visual perception data, and coordinate the various modules through the Internet of Things (IoT). For example, they use secure transmission technology to transmit data between modules and reduce transmission latency through dynamic routing optimization.

[0077] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An IoT-based intelligent control system for elderly care robots, comprising a multimodal perception layer and a system linkage module. The multimodal perception layer is used to perform data acquisition operations to obtain multimodal data packets; the system linkage module is used to store force perception data, voice perception data, and visual perception data, and coordinates the various modules through the Internet of Things. Its features are, Also includes: The safety decision-making center includes a force control decision-making unit and a safety scheduling unit; The force control decision unit is used to receive multi-mode data packets, perform behavioral risk assessment and analysis on the multi-mode data packets to obtain the total risk value, and then transmit the total risk value to the security scheduling unit; The safety scheduling unit is used to perform scheduling analysis on the total risk value to obtain the corresponding scheduling instructions, and then send the scheduling instructions to the execution driver module; The execution driver module is used to acquire the intent of the voice command and drive its execution according to the intent of the voice command. It also receives scheduling instructions, identifies the corresponding specific execution items, and drives the execution according to the specific execution items. At the same time, it starts the voice interaction module to conduct voice interaction with the user, acquire the user's voice information, and transmit the voice information to the security decision center. The voice interaction optimization module is used to receive multi-mode data packets, perform voice interaction optimization analysis on the multi-mode data packets to obtain optimization strategies, optimize the command intent based on the optimization strategies to obtain voice command intents, and send the voice command intents to the execution driver module.

2. The intelligent control system for elderly care robots based on the Internet of Things according to claim 1, characterized in that, The specific analysis method for conducting behavioral risk assessment and analysis on multi-mode data packets is as follows: The current force perception data, voice perception data, and view perception data are obtained based on the multi-mode data packet; the contact force, displacement, and mechanical joint angle data of each robotic arm are obtained based on the force perception data. The displacement deviation is obtained by calculating the difference between the displacement and the corresponding preset displacement target. The contact force of the robotic arm and the displacement deviation are then substituted into the hyperbolic tangent function coupling model to calculate the displacement force coupling risk value. The preset action execution cycle is obtained based on the mechanical joint angle data, and the action execution cycle is divided into multiple action monitoring points based on the preset acquisition interval. The joint angle offset vector corresponding to each action monitoring point is obtained based on the mechanical joint angle data, and the offset angle risk value is calculated using the improved Lyapunov exponential spectrum analysis formula. The mechanical risk value is obtained by summing the displacement force coupling risk value and the offset angle risk value at the current moment. Based on the current corresponding speech perception data and view perception data, obtain speech command information and behavioral body information; Based on the voice command information, the intention of the elderly voice command is obtained, and the intention of the voice command is matched with the risk fuzzy command library pre-stored in the database. When the intention of the voice command matches the risk fuzzy command library, the corresponding intention of the voice command is marked as a risk voice command. The confidence level of the voice command is calculated by averaging the number of risk voice commands corresponding to the current analysis period with the total number of corresponding voice command intentions. The amount of change in the elderly’s movements is obtained by obtaining behavioral and bodily information. The amount of change in movements is calculated with a preset base unit time to obtain the rate of change in movements. The behavioral risk value is calculated using a preset posture-voice exponential function fusion model. Environmental data is obtained through view perception data. Obstacle data and ground flatness are obtained based on the environmental data. The distance to each obstacle is obtained based on the obstacle data. The difference between the ground flatness and the preset flatness benchmark is calculated to obtain the ground flatness deviation value. The environmental risk value is calculated using the formula of the distance attenuation correction model. The mechanical risk value, behavioral risk value, and environmental risk value are all normalized and their values ​​are taken. The total risk value is calculated by substituting the normalized values ​​into the formula of the nonlinear fusion model.

3. The intelligent control system for elderly care robots based on the Internet of Things according to claim 2, characterized in that, The specific analysis method for scheduling the total risk value is as follows: Let the current total risk value be G, and obtain the pre-set risk classification value G. min G high G max Based on each risk classification value, the risk classification interval [0, G] is obtained. min ]、(G min G high ]、(G high G max ]、(G max ,∞), match the current total risk value with each risk level value, when G∈[0,G min When G∈(G), the risk state is generated as normal operation; when G∈(G) min G high When G∈(G), the risk status is low, and the corresponding response measures are early warning prompts and voice inquiries; high G max When G∈(G), the risk status is medium risk, and the corresponding response measures are downgraded operation and voice inquiry; max When the risk value is ∞, the risk status is high risk and the corresponding response measure is emergency termination; the emergency response measure corresponding to the current total risk value and the risk status is marked as a dispatch instruction.

4. The intelligent control system for elderly care robots based on the Internet of Things according to claim 1, characterized in that, The specific analysis method for optimizing voice interaction using multi-mode data packets is as follows: Voice interaction optimization analysis obtains the multi-mode data packets corresponding to the current voice interaction, and the voice interaction analysis includes sub-fuzzy quantification analysis, comprehensive fuzziness evaluation, and intent optimization correction; Subfuzzy metric analysis analyzes multimode data packets to obtain multiple subfuzzy metric values ​​corresponding to the current voice interaction; The comprehensive fuzziness assessment obtains comprehensive fuzziness assessment information by comprehensively analyzing the quantified values ​​of sub-fuzzy metrics; Intent optimization correction generates an intent optimization strategy based on comprehensive ambiguity evaluation information, and the intent optimization is performed through the intent optimization strategy to obtain the intent of the voice command.

5. The intelligent control system for elderly care robots based on the Internet of Things according to claim 4, characterized in that, The specific analysis content of the sub-fuzzy quantitative analysis is as follows: Force perception data, speech perception data, and view perception data are obtained based on the multi-mode data packets corresponding to the current voice interaction. Speech text recognition information and speech rate information corresponding to each speech information of the user are obtained based on the speech perception data. The current speech rate of the user is obtained based on the speech rate information. The speech rate deviation is calculated by comparing the current speech rate of the user with the average speech rate in the preset time period. The text recognition degree and semantic integrity score are obtained based on the speech text recognition information. The speech ambiguity is calculated using the formula of multi-dimensional feature weighting method. A preset speech ambiguity threshold is obtained. When the speech ambiguity is greater than the speech ambiguity threshold, the part of the speech ambiguity exceeding the speech ambiguity threshold is marked as the speech sub-ambiguity quantification value. Based on the database, the mean and standard deviation of force feedback corresponding to the user's daily cooperation are obtained. The current feedback force is obtained based on the force perception. The force ambiguity is calculated based on the formula of the normal distribution-dynamic weight model. Obtain the preset upper limit of force ambiguity. When the force ambiguity is greater than the upper limit, mark the corresponding force ambiguity as the force sub-ambiguity quantification value. Based on view-aware data, facial expression information and lip movement features corresponding to each user's voice information are obtained. The facial expression information and lip movement features are matched with the corresponding voice information to identify the matching degree. The matching degree of facial expression command and lip movement command are obtained through recognition output. The matching degree is the set of historical facial expression information and the set of historical lip movement features corresponding to each voice information. When there is no match with the corresponding set, the matching degree of the corresponding command is zero. When there is a match with the corresponding set, the matching degree of the corresponding command is the matching value given by the preset design. The facial expression command matching degree and lip movement command matching degree are used to calculate the view ambiguity using the multimodal matching algorithm formula, and the view ambiguity is marked as the view sub-ambiguity quantification value.

6. The intelligent control system for elderly care robots based on the Internet of Things according to claim 5, characterized in that, The comprehensive ambiguity assessment is specifically conducted as follows: the fuzzy quantification values ​​of the speech sub-fuzzy quantification value, the fuzzy quantification value of the force sub-fuzzy quantification value, and the fuzzy quantification value of the view sub-fuzzy quantification value are all normalized and their values ​​are taken. The comprehensive ambiguity value is then calculated by substituting each fuzzy quantification value into the formula of the weighted correction-membership degree conversion method.

7. The intelligent control system for elderly care robots based on the Internet of Things according to claim 6, characterized in that, The intention optimization and correction specifically involves: obtaining a pre-set fuzziness level classification value; matching the current corresponding comprehensive fuzziness with each fuzziness level classification value to obtain the corresponding fuzziness level and optimization strategy; The levels of fuzziness include low fuzziness, medium fuzziness, high fuzziness, and extremely high fuzziness; The optimization strategies corresponding to each fuzziness level include directly using speech recognition results, using speech results as the basis and supplemented by visual data, using medium fuzziness as the basis for optimization and combining force data for verification, and ignoring speech commands and taking force and visual data as the main factors.

8. The intelligent control system for elderly care robots based on the Internet of Things according to claim 1, characterized in that, The multimodal perception layer specifically involves: acquiring force perception data, voice perception data, and view perception data through multimodal sensors; combining the force perception data, voice perception data, and view perception data to obtain a multimodal data packet; and sending the multimodal data packet to the security decision center and the voice interaction module.

9. The intelligent control system for elderly care robots based on the Internet of Things according to claim 8, characterized in that, The force perception data includes the contact force, displacement, mechanical joint angle data, and current feedback force of each robotic arm; the voice perception data includes voice command information and behavioral body information, as well as the voice text recognition information and speech rate information corresponding to each voice message of the user; the view perception data includes obstacle data and environmental data on ground flatness, as well as the facial expression information and lip movement features corresponding to each voice message of the user.

Images