User vision art education assisting operation method, robot system and interaction system

Abstract

The invention discloses an auxiliary user vision art education operation method, a robot system and an interaction system, the auxiliary user vision art education operation method is applied to the robot system, the robot system comprises an interaction interface and a mechanical arm, at least part of the mechanical arm is arranged on the hand of a user in a sleeving mode, and the method comprises the steps that based on the ability test result of the user, the interaction interface is obtained; creating an artistic creation task suitable for the user; displaying guide information corresponding to the artistic creation task on the interactive interface; the guide information is used for guiding the user to perform creation operation on the interactive interface; and in the creation process of the user, the mechanical arm is controlled to assist the hand movement of the user, and the art works related to the art creation task are obtained. At least part of the mechanical arm can be arranged on the hand of a user in a sleeving mode so as to assist the hand of the user. According to the guiding information set by the artistic creation task, the artistic creation task can be simplified, gradual guiding of user creation is realized, and the creation difficulty is reduced.

Description

Technical Field

[0001] This application relates to the field of robotics technology, and in particular to methods, robotic systems, and interactive systems that assist users in visual arts education. Background Technology

[0002] As people's living standards improve, various service robots have emerged to meet the diverse spiritual and cultural needs of users. However, current service robots on the market are designed for the general public, and their functions and structures are not suitable for elderly or hand-impaired users. In other words, service robots on the market are poorly targeted at elderly or hand-impaired users and cannot meet the needs of this group. Summary of the Invention

[0003] This application provides a method, robot system, and interactive system for assisting users in visual arts education. At least a portion of a robotic arm can be fitted over the user's hand to provide assistance. Furthermore, the robotic arm can provide assistance throughout the entire creative process, or it can provide assistance only when the user's movements are detected to be unstable. This method of providing assistance only when unstable reduces the difficulty of creation while preserving user autonomy, avoiding the dependency problem caused by full-process assistance. Based on the guidance information set in the art creation task, the art creation task can be simplified, enabling gradual guidance for the user's creation and reducing the difficulty of creation.

[0004] To address the aforementioned technical problems, this application provides a method for assisting users in visual arts education, applied to a robotic system. The robotic system includes an interactive interface and a robotic arm, with at least a portion of the robotic arm fitted onto the user's hand. The method for assisting users in visual arts education includes: creating an art creation task suitable for the user based on the user's ability test results; displaying guidance information corresponding to the art creation task on the interactive interface; the guidance information guiding the user to perform creative operations on the interactive interface; and controlling the robotic arm to assist the user's hand movements during the user's creation process to obtain an artwork related to the art creation task.

[0005] In some embodiments, before controlling a robotic arm to assist the user's hand movements to obtain an artwork related to the art creation task during the user's creation process, the method for assisting the user's visual arts education operation further includes: acquiring the user's hand trajectory information; and generating a guidance path corresponding to the art creation task and / or guidance information based on the hand trajectory information.

[0006] During the user's creation process, the robotic arm is controlled to assist the user's hand movements to obtain artworks related to the artistic creation task. This includes: during the user's creation process, the robotic arm is controlled to assist the user's hand movements according to a guided path so that artworks related to the artistic creation task can be obtained on the interactive interface.

[0007] In some embodiments, during the process of controlling a robotic arm to assist a user's hand movements, the method for assisting a user in visual arts education operations includes: acquiring first information from the user; wherein the first information includes at least one of force and speed detected in the interactive interface during the creation process; and adjusting the robotic arm's assistance mode for the user based on the first information.

[0008] In some embodiments, during the user's creative process, the method for assisting the user in visual arts education further includes: during the formation of the artwork, in response to a discrepancy between the current data of the artwork and the current guidance data in the guidance information, displaying a prompt message on the interactive interface; the prompt message is used to remind the user to adjust the artwork according to the current guidance data.

[0009] In some embodiments, during the user's creative process, the method for assisting the user in visual arts education further includes: detecting second information of the user; the second information includes at least one of the user's blood oxygen and heart rate detected by the robotic arm during the creative process; when the second information does not meet preset conditions, issuing a warning message, and / or controlling the robotic arm to stop assisting the user's hand; wherein the warning message is configured to remind the user to stop creating.

[0010] In some embodiments, the robot system further includes a voice module;

[0011] When the second information does not meet the preset conditions, a warning message is issued, including: when the second information does not meet the preset conditions, controlling the interactive interface to display the warning message, and / or controlling the voice module to broadcast the warning message.

[0012] In some embodiments, before creating an art creation task suitable for the user based on the user's ability test results, the method for assisting the user in visual arts education further includes: detecting second information about the user; the second information includes at least one of the user's blood oxygen and heart rate detected by the robotic arm during the creation process; and when the second information meets preset conditions, performing the step of creating an art creation task suitable for the user based on the user's ability test results.

[0013] In some embodiments, the robot system interacts with an interactive platform;

[0014] After obtaining the artwork related to the art creation task, the method for assisting users in visual arts education further includes: uploading the artwork to the interactive platform; and / or, after uploading the artwork to the interactive platform, receiving feedback information sent by the interactive platform.

[0015] In some embodiments, before the user creates, the method for assisting the user in visual arts education further includes: adjusting the display information of the interactive interface based on the results of the ability test so that the display information of the interactive interface is suitable for the user; the adjustment includes at least one of font adjustment, layout adjustment and color adjustment.

[0016] To address the aforementioned technical problems, this application also provides a robot system, including an interactive interface, a robotic arm, a voice module, a memory, and a controller. The memory stores a computer program, and the controller controls the robotic arm, the interactive interface, and the voice module based on the computer program to implement the aforementioned method for assisting users in visual arts education.

[0017] To address the aforementioned technical problems, this application also provides an interactive system, including an interactive platform and the aforementioned robot system. The robot system is communicatively connected to the interactive platform and is used to: upload artworks to the interactive platform; and / or, after uploading artworks to the interactive platform, receive feedback information sent by the interactive platform.

[0018] The user-assisted visual arts education operation method, robot system, and interactive system provided in some embodiments of this application can fit at least a portion of a robotic arm onto the user's hand to assist the user's hand. Furthermore, the robotic arm can provide assistance throughout the entire creation process, or it can provide assistance only when the user's movements are detected to be unstable. Providing assistance only when unstable reduces the difficulty of creation while preserving user autonomy, avoiding the dependency problem caused by full-process assistance. Based on the guidance information set in the art creation task, the art creation task can be simplified, enabling gradual guidance for the user's creation and reducing the difficulty of creation. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0020] Figure 1 This is a flowchart illustrating the user-assisted visual arts education operation method provided in some embodiments of this application;

[0021] Figure 2 This is a flowchart illustrating the user-assisted visual arts education operation method provided in some embodiments of this application;

[0022] Figure 3 This is a flowchart illustrating the user-assisted visual arts education operation method provided in some embodiments of this application;

[0023] Figure 4 This is a flowchart illustrating the user-assisted visual arts education operation method provided in some embodiments of this application;

[0024] Figure 5 These are schematic diagrams of the robot system provided in some embodiments of this application;

[0025] Figure 6 This is a schematic diagram of the structure of the interactive system provided in some embodiments of this application. Detailed Implementation

[0026] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0027] According to some embodiments of this application, please refer to Figures 1 to 4 , Figures 1 to 4 These are schematic flowcharts illustrating the user-aided visual arts education operation methods provided in some embodiments of this application.

[0028] Please see Figures 1 to 2 This application describes a method for assisting users in visual arts education, applied to a robotic system. The robotic system includes an interactive interface and a robotic arm, at least a portion of which is fitted onto the user's hand. The robotic arm involved in this application is a multi-degree-of-freedom (DOF) robotic arm (e.g., a 6-DOF or 7-DOF robotic arm), where each DEF represents an independent joint capable of generating an independent movement. The multi-DOF robotic arm supports safe human-machine collaboration. The portion of the robotic arm fitted onto the user (i.e., the end effector) is a wearable component (e.g., a wristband or other material), which can be made of flexible materials such as silicone or plastic to avoid injury to the user's hand. Furthermore, it can have built-in or integrated sensors (e.g., a heart rate sensor), allowing real-time monitoring of the user's physiological state and posture without relying on external wristbands or watches, providing timely rest reminders or safety interventions, effectively ensuring the user's health and safety during the creative process.

[0029] like Figure 1 As shown, the methods for assisting users in visual arts education include:

[0030] Step 11: Based on the user's ability test results, create an art creation task suitable for the user.

[0031] In some embodiments, the robot system includes an ability testing device configured to test the user's ability and obtain an ability test result. The users involved in this application can be elderly users, young users or juvenile users, for use in assisting the visual art education of elderly users, young users or juvenile users.

[0032] The ability test involved in this application is related to the user's visual sensitivity, hearing, hand flexibility, etc. The ability test result obtained after the ability test can be directly used to create an art creation task suitable for the user; or, the ability test result is stored in a memory, server, cloud or other devices, and the ability test result can be obtained from devices such as the memory, server, cloud, etc. before creating an art creation task suitable for the user.

[0033] It can be understood that the ability test results of different users are different. A real-time ability test can be performed on the user each time the robot system is used to ensure the real-time update of the ability test result and avoid errors in the assistance of the robotic arm and other components in the robot system to the user; here, it is not required that the user log in to an account, etc., which is convenient for different users to use at any time. Or, when the user uses the robot system, according to the user information (account number, account name, etc.) input by the user on the interaction interface, the ability test result corresponding to the user is obtained from a memory, server, cloud or other devices, avoiding repeated detections and saving time.

[0034] The art creation task involved in this application is related to the user's visual sensitivity, hearing, hand flexibility, etc. For example, when the user's visual sensitivity is poor, an art creation task related to calligraphy can be generated; another example is that when the user's hands are not flexible, an art creation task related to a painting target with fewer details and thicker lines can be generated.

[0035] In some embodiments, the art creation task suitable for the user can be obtained by making targeted modifications to a template creation task according to the ability test result, where the template creation task is set in advance. For example, the template creation task is related to calligraphy and is to write the character "Fu". When the ability test result of the user shows that the user's hands are not flexible, the writing target can be modified from writing the character "Fu" to writing the character "Da" with fewer strokes to reduce the difficulty.

[0036] In some embodiments, multiple template creation tasks are displayed on the interaction interface. After the user interacts with the interaction interface and selects a target template creation task, the robot system can call the user's ability test result to make targeted modifications to the target template creation task, and then obtain an art creation task suitable for the user.

[0037] In one application scenario, the capability testing device can perform vision tests and hand dexterity tests when the robot system is started. The test results are used to initialize and set fixed parameters, such as automatically setting large font size and high contrast color scheme for those with poor vision. The hand dexterity test results are used to determine the threshold of the robotic arm's auxiliary force. In subsequent use, the layout of the interactive interface and font parameters remain static.

[0038] The ability test involved in this application can be related to the user's creative level. The user's creative level can be obtained directly from user information; or it can be determined by the user's answers to a questionnaire displayed on the interactive interface. The questionnaire may include information such as the user's gender, age, mood, art interests, whether they have basic drawing skills, and whether they have basic calligraphy skills.

[0039] Therefore, the art creation tasks involved in this application are related to the user's creative level. For example, when the user has basic painting skills, art creation tasks related to traditional Chinese painting, watercolor painting, etc. can be generated; when the user has basic calligraphy skills, art creation tasks related to calligraphy can be generated; when the user is female, art creation tasks that are of interest to women can be generated; and when the user is in a bad mood, art creation tasks with lower difficulty can be generated.

[0040] In some embodiments, an art creation task suitable for the user can be generated by combining the user's creative level with the user's visual sensitivity, hearing and hand dexterity (ability test results).

[0041] Step 12: Display the guidance information corresponding to the art creation task on the interactive interface; the guidance information is used to guide the user to perform creative operations on the interactive interface.

[0042] The robot system described in this application can also embed an AI (Artificial Intelligence) interaction module into the interactive interface. This AI interaction module can generate guidance information corresponding to the artistic creation task and display the generated guidance information on the interactive interface. The guidance information can simplify complex artistic creation tasks into step-by-step operations, guiding users to complete the artwork progressively.

[0043] The guidance information can be displayed gradually based on the user's real-time creative data on the interactive interface, so that the guidance information corresponds to the user's creative process; or, all the guidance information can be displayed on the interactive interface by sliding left and right or up and down, so that the user can preview the guidance information for the next step of creation and make it easier for the user to adjust the creative layout on the interactive interface.

[0044] In one application scenario, the interactive interface can display the creation steps based on the preset decomposition of the AI ​​interaction module. For example, a calligraphy work can be decomposed into fixed stroke steps such as dot, horizontal stroke, left-falling stroke, and right-falling stroke.

[0045] Step 13: During the user's creation process, control the robotic arm to assist the user's hand movements and obtain an artwork related to the artistic creation task.

[0046] In some embodiments of this application, the user-assisted visual arts education operation method provides a step-by-step demonstration of preset creation steps combined with robotic arm guidance, which can simplify complex art creation tasks into operable logical units.

[0047] In some embodiments, before controlling the robotic arm to assist the user's hand movements to obtain an artwork related to the artistic creation task, the method for assisting the user in visual arts education further includes: detecting second information of the user; the second information includes at least one of the user's blood oxygen and heart rate detected by the robotic arm during the creation process; when the second information meets preset conditions, executing the step of controlling the robotic arm to assist the user's hand movements to obtain an artwork related to the artistic creation task.

[0048] In some embodiments, the robot system includes a physiological parameter detection module, which is disposed on the robotic arm (such as the inner wall of the silicone ring at the end of the robotic arm or the end effector of the robotic arm). This eliminates the need for additional external devices such as wristbands or watches, thus avoiding the discomfort and cumbersome operation caused by external wristbands and improving the ease of use and comfort of the robot system.

[0049] When the robotic arm is at least partially fitted onto the user's hand, the sensors included in the physiological parameter detection module directly contact the user's skin to detect the user's physiological parameters; here, the detection of physiological parameters can be continuous. Specifically, the robotic arm includes a robotic arm body, and the physiological parameter detection module is located on the robotic arm body. When the robotic arm body is at least partially fitted onto the user's hand, it automatically detects the user's physiological parameters. It can be understood that real-time monitoring of the user's physiological state can promptly issue health alerts, rest reminders, or safety interventions, effectively ensuring the user's health and safety during the creative process.

[0050] In some embodiments, the physiological parameter detection module includes a blood oxygen detection unit, a heart rate detection unit, and / or a temperature detection unit.

[0051] The blood oxygen detection unit is used to detect the user's blood oxygen; the blood oxygen detection unit includes a blood oxygen sensor; for example, the blood oxygen detection unit can use a photoelectric sensor to detect blood oxygen and calculate the user's blood oxygen saturation by measuring the light absorption rate.

[0052] The heart rate detection unit is used to detect the user's heart rate. The heart rate detection unit includes a heart rate sensor. For example, the heart rate detection unit can use a photoplethysmographic (PPG) sensor to detect heart rate by monitoring changes in blood flow at the wrist to obtain the user's heart rate.

[0053] The temperature detection unit is used to detect the user's body temperature. The temperature detection unit includes a temperature sensor. For example, the temperature detection unit can use a thermistor or an infrared temperature sensor to detect temperature and directly measure the user's skin surface temperature.

[0054] In other embodiments, the physiological parameter detection module may include a skin conductance detection unit for detecting the user's current skin conductance, and the skin conductance detection unit includes a skin conductance sensor.

[0055] When the physiological parameter detection module includes at least two detection units (such as blood oxygen detection unit and heart rate detection unit, blood oxygen detection unit and temperature detection unit, blood oxygen detection unit, heart rate detection unit and temperature detection unit), at least two detection units should be detected simultaneously to save detection time; or, at least two detection units should be detected sequentially to avoid obtaining detection results at the same time, which could lead to missing some detection results when the user obtains multiple detection results, or the user obtaining the wrong detection results, etc.

[0056] In some embodiments, the robot system includes a reminder module, which generates a reminder message when physiological parameters do not meet preset conditions; the reminder message is configured to remind the user to stop creating. Since the physiological parameter detection module includes a blood oxygen detection unit, a heart rate detection unit, and / or a temperature detection unit, the preset conditions for different detection units are different. For example, the preset condition for the heart rate detection unit is 60 beats / min to 120 beats / min; if the heart rate is not within the range of 60 beats / min to 120 beats / min, it is determined that the user's heart rate exceeds the preset safety range and does not meet the preset conditions. Similarly, the preset condition for the blood oxygen detection unit is 95% to 100%; if the blood oxygen level is not within the range of 95% to 100%, it is determined that the user's blood oxygen level exceeds the preset safety range and does not meet the preset conditions.

[0057] In some embodiments, the reminder module includes a voice reminder unit and / or an indicator light reminder unit.

[0058] The voice prompt unit is used to broadcast voice messages when physiological parameters do not meet preset conditions; or to broadcast guidance information during the user's artistic creation process; or to issue a prompt sound indicating completion of the artwork after the artwork is obtained.

[0059] It's worth noting that the voice reminder unit prioritizes voice broadcasts, giving priority to broadcasting alerts when physiological parameters do not meet preset conditions. The content of the voice broadcasts can be related to specific physiological parameters, such as "User's heart rate is abnormal, please rest," "User's heart rate is too low, please rest," or "User's heart rate is too high, please rest." In addition to broadcasting voice messages, the voice reminder unit can also record audio, which can be used for user account login, issuing commands, or other purposes.

[0060] The indicator light unit is used to illuminate a light when physiological parameters do not meet preset conditions; or, illuminate a target color light when physiological parameters do not meet preset conditions, with different target colors corresponding to different physiological parameters (heart rate, temperature, and blood oxygen); or, illuminate a light in a non-continuous manner when physiological parameters do not meet preset conditions; or, otherwise, no restrictions are imposed here.

[0061] In other embodiments, the reminder module can also generate notification information when physiological parameters meet preset conditions. For the voice reminder unit, this could be: "User heart rate is normal" when the user's heart rate is normal, "User body temperature is normal" when the user's body temperature is normal, "User blood oxygen is normal" when the user's blood oxygen is normal, or other notifications. For the indicator light reminder unit, this could be: illuminating a light of a color other than the target color when physiological parameters meet preset conditions, or continuously illuminating the light when physiological parameters meet preset conditions, or other notifications.

[0062] To emphasize the importance of information and avoid interference, notification messages are typically not generated when physiological parameters meet preset conditions during the user's creative process on the interactive interface. Instead, a physiological parameter detection module can be used to detect the user's physiological parameters before they begin creating on the interactive interface, during user ability testing, or before an art creation task is assigned. A notification message can then be generated when these physiological parameters meet preset conditions. This notification message is configured to indicate that the user's physical condition supports their ability to use or continue using the robot system for art creation.

[0063] In some embodiments, since the physiological parameter detection module includes a blood oxygen detection unit, a heart rate detection unit, and / or a temperature detection unit, etc., here:

[0064] When the physiological parameter detection module includes only one detection unit, the system can choose to use either a voice alert or an indicator light alert to remind the user that the physiological parameter does not meet the preset conditions, depending on the actual situation. The goal is to make the alert information prominent and easier for the user to understand accurately. For example, in a noisy environment, to avoid the user receiving incorrect or unreceived information, an indicator light alert can be used. Similarly, in environments with complex colors or in quiet environments, a voice alert can be used. In environments with simple colors and in quiet environments, both an indicator light alert and a voice alert can be used. Furthermore, when the detected heart rate value is consistently higher than a set threshold, the robot system can activate the voice alert unit to broadcast the message and simultaneously activate the indicator light to flash, ensuring that the alert information can be received by users with different sensory abilities.

[0065] When the physiological parameter detection module includes at least two detection units, if the at least two detection units generate multiple reminder messages, the reminder messages can be issued sequentially using only one of the voice reminder unit and the indicator light reminder unit. The order in which the multiple reminder messages are issued is determined according to the degree of abnormality, or according to the detection order of different detection units. Alternatively, the voice reminder unit and the indicator light reminder unit can be used to issue multiple reminder messages separately, with the reminder messages issued by the voice reminder unit and the indicator light reminder unit determined according to the actual situation (such as according to the environmental factors mentioned above). Or, the voice reminder unit and the indicator light reminder unit can be used to issue multiple reminder messages separately. For example, the reminder message can be broadcast first using the voice reminder unit, then the reminder message can be issued using the indicator light reminder unit, and then the above order can be repeated.

[0066] For example, when blood oxygen levels do not meet the first preset condition and heart rate does not meet the second preset condition, the voice prompt unit sequentially announces "User's blood oxygen is abnormal" and "User's heart rate is abnormal." Alternatively, the indicator light unit first emits a red light to indicate "User's blood oxygen is abnormal," and then emits a green light to indicate "User's heart rate is abnormal." Or, when blood oxygen levels do not meet the first preset condition and heart rate does not meet the second preset condition, the voice prompt unit announces "User's blood oxygen is abnormal," and the indicator light unit emits a green light to indicate "User's heart rate is abnormal." Because two different prompt units are used, the announcements "User's blood oxygen is abnormal" and "User's heart rate is abnormal" are displayed. The "abnormal" alerts can be issued simultaneously or sequentially without interfering with each other. However, if the body temperature does not meet the third preset condition at the same time, and the voice alert unit or indicator light alert unit needs to issue the "user temperature abnormal" alert, the two alerts issued by the voice alert unit or indicator light alert unit have a specific order. For example, the voice alert unit may announce "user blood oxygen abnormal" and "user temperature abnormal". The two alerts are issued sequentially. If the blood oxygen abnormality is more severe, "user blood oxygen abnormal" will be announced first. If the temperature abnormality is more severe, "user temperature abnormal" will be announced first.

[0067] In some embodiments, the reminder information generated by the reminder module can also be displayed on the interactive interface. Specifically, it can be displayed on the interactive interface without affecting the user's artistic creation and guidance information display, and without affecting the position where the user focuses on the reminder information.

[0068] In some embodiments of this application, the reminder module includes a voice reminder unit and / or an indicator light reminder unit. By monitoring the user's physiological state in real time and triggering multimodal reminders when abnormalities occur, the reminder module can address the lack of health and safety intervention in existing elderly-assistance robots, reducing the risk of fainting or discomfort caused by health abnormalities (such as high heart rate) during artistic creation in elderly users. The dual reminder method of voice and indicator lights can adapt to the sensory differences of elderly users, ensuring that users with hearing or vision limitations can receive timely warnings, avoiding safety hazards caused by continuing creation due to an inability to perceive health problems. Simultaneously, this mechanism requires no active user operation or external devices, being directly integrated into the physiological parameter detection module. This can improve the safety and age-friendliness of the robot system, extend the user's safe creation time, and enhance the user's dependence on and confidence in using the robot system.

[0069] like Figure 2 As shown, the methods for assisting users in visual arts education include:

[0070] Step 21: Based on the user's ability test results, create an art creation task suitable for the user.

[0071] Step 22: Display guidance information corresponding to the art creation task on the interactive interface; the guidance information is used to guide the user to perform creative operations on the interactive interface.

[0072] Step 23: Obtain the user's hand trajectory information.

[0073] In some embodiments, the robot system further includes an image acquisition device; the image acquisition device includes an RGB (Red, Green, Blue) camera, a depth sensor, or a binocular camera, etc.; the image acquisition device is used to acquire images of the user in real time.

[0074] In some embodiments, the images acquired by the image acquisition device may include images of the user's hand. Here, a robotic arm is connected to the image acquisition device. The robotic arm can process the user's hand image to obtain the user's hand trajectory information, and then generate a guidance path corresponding to the art creation task and / or guidance information based on the hand trajectory information. The robotic arm assists the user's hand movement according to the guidance path so as to obtain an artwork related to the art creation task on the interactive interface.

[0075] Specifically, the robotic arm includes a first processing unit and a robotic arm body; the first processing unit is connected to an image acquisition device; the robotic arm body is connected to the first processing unit, and the robotic arm body is at least partially fitted onto the user's hand. The first processing unit may be a processor, which is connected to the image acquisition device to analyze image data and identify the user's motion state (such as tremors) and / or motion trajectory; the robotic arm body may be a flexible medical-grade silicone ring, at least partially fitted onto the user's wrist, forming a wearable design to achieve direct physical contact with the user's hand, ensuring effective transmission of assistive force without interference from external devices, avoiding discomfort caused by wearing additional devices, and significantly enhancing the operational comfort and creative safety of elderly users, etc.

[0076] Specifically, the first processing unit can process the user's hand image to obtain the user's hand trajectory information; further, the first processing unit can also process the hand trajectory information to generate a guidance path corresponding to the art creation task and / or guidance information based on the hand trajectory information. During the user's creation process, the robotic arm can assist the user's hand movement according to the guidance path to obtain an artwork related to the art creation task on the interactive interface.

[0077] In other embodiments, the images acquired by the image acquisition device may include user face images, which can be used for face recognition. When face recognition is successful, the user's account associated with the face can be logged in on the interactive interface.

[0078] In some embodiments, the robotic system includes a posture sensor and a depth sensor (corresponding to an image acquisition device). The posture sensor, combined with the depth sensor, analyzes the user's sitting posture and hand movements, enabling the detection of incorrect postures and triggering safety prompts. Here, real-time monitoring of the user's hand and other postures allows for timely health alerts, adjusting the robotic arm's assistance mode for the user.

[0079] Step 24: Based on hand trajectory information, generate a guidance path corresponding to the artistic creation task and / or guidance information.

[0080] Step 25: During the user's creation process, control the robotic arm to assist the user's hand movements according to the guided path, so as to obtain an artwork related to the art creation task on the interactive interface.

[0081] Please see Figures 3 to 4 The method of assisting users in visual arts education is applied to a robot system. The robot system includes an interactive interface and a robotic arm. At least part of the robotic arm is fitted onto the user's hand, and the robot system interacts with the interactive platform. Here, the interactive platform can be one of the following: cloud, server, or client (such as a laptop, smartphone, or smart wearable device).

[0082] like Figure 3 As shown, the methods for assisting users in visual arts education include:

[0083] Step 31: Based on the user's ability test results, create an art creation task suitable for the user.

[0084] Step 32: Display guidance information corresponding to the art creation task on the interactive interface; the guidance information is used to guide the user to perform creative operations on the interactive interface.

[0085] Step 33: During the user's creation process, control the robotic arm to assist the user's hand movements and obtain an artwork related to the artistic creation task.

[0086] Step 34: Upload the artwork to the interactive platform.

[0087] like Figure 4 As shown, the methods for assisting users in visual arts education include:

[0088] Step 41: Based on the user's ability test results, create an art creation task suitable for the user.

[0089] Step 42: Display guidance information corresponding to the art creation task on the interactive interface; the guidance information is used to guide the user to perform creative operations on the interactive interface.

[0090] Step 43: During the user's creation process, control the robotic arm to assist the user's hand movements and obtain an artwork related to the artistic creation task.

[0091] Step 44: Upload the artwork to the interactive platform.

[0092] Step 45: Receive feedback information sent by the interactive platform.

[0093] Feedback information can include comments on artworks or other feedback.

[0094] According to some embodiments of this application, in the process of controlling the robotic arm to assist the user's hand movement, the method of assisting the user in visual arts education operations further includes: obtaining first information of the user; the first information includes at least one of force and speed detected in the interactive interface during the creation process; and adjusting the assist mode of the robotic arm for the user based on the first information.

[0095] In some embodiments, the robot system includes a force detection module for real-time detection of the force applied by the user on the interactive interface during the artistic creation process; the force detection module includes a tactile feedback sensor or a force feedback sensor, etc.

[0096] The force detection module can be integrated into the inner wall of the silicone ring at the end of the robotic arm. Specifically, a tactile feedback sensor or a force feedback sensor can be placed on the inner surface of the silicone ring. When the user slips their wrist into the silicone ring to practice calligraphy or painting, the tactile feedback sensor or force feedback sensor can detect the pressure changes generated by the contact between the user's hand and the robotic arm.

[0097] The resistance of the user's hand can be determined based on the detection results (i.e., force data) from the force detection module. The amplitude of the user's hand tremors can be dynamically analyzed to determine whether the robotic arm's assistance mode needs adjustment, thus achieving dynamic adjustment of the robotic arm's assistance force. For example, if excessive force is detected from the user's hand, the guiding force can be automatically reduced to prevent excessive force and damage to the interactive interface; or if insufficient force is detected, the assistance can be enhanced to support the user in completing creative actions, ensuring that artistic handwriting is left on the interactive interface.

[0098] The force detection module provided in this application can form a closed-loop feedback system with the robotic arm, eliminating the need for external devices such as wristbands or watches. Force detection is achieved directly through sensors integrated into the wearable component. Real-time monitoring of the force applied by the user and dynamic adjustment of the robotic arm's assistance force effectively solves the practice obstacles faced by elderly users or those with hand-eye coordination issues due to unstable hand strength or decreased coordination during artistic creation. It avoids fatigue caused by excessive assistance or creative failure due to insufficient assistance, significantly improving the safety and smoothness of the creative process. Simultaneously, this adaptive adjustment mechanism enhances the operational comfort and learning confidence of elderly users, reduces practice interruptions or health risks caused by discomfort, extends user practice time, and enhances learning motivation, thereby improving sustained participation and learning outcomes in artistic creation or art education.

[0099] In some embodiments, the robot system includes a speed detection module for real-time detection of the user's creative speed during the artistic creation process; the speed detection module includes a posture sensor or a gyroscope sensor, etc.

[0100] Based on the speed detection module's results, it can be determined whether the robotic arm's assistance mode for the user's hand needs adjustment. This includes adjusting the speed at which the robotic arm assists the user's hand movements to prevent the user's creation speed from being too fast or too slow, which could prevent the robotic arm from providing adequate assistance. The initial assistance mode of the robotic arm can be determined based on capability test results, or the user can select the initial assistance mode through interaction with the interface.

[0101] According to some embodiments of this application, the robot system based on any of the above embodiments, in the user's creative process, further includes the following method for assisting the user in visual arts education: during the formation of the artwork, in response to a discrepancy between the current data of the artwork and the current guidance data in the guidance information, displaying a prompt message on the interactive interface; the prompt message is used to remind the user to adjust the artwork according to the current guidance data. This adjustment may include font adjustments (e.g., adjusting to a larger font), line adjustments (e.g., adjusting to thicker lines), layout adjustments (e.g., adjusting to the center position), and color adjustments (e.g., adjusting to a color with significant contrast).

[0102] It is understandable that adjusting artworks can significantly improve visual readability, reduce user fatigue, and prevent the elderly, people with poor eyesight, and other groups from repeatedly adapting to the interface.

[0103] According to some embodiments of this application, the robot system based on any of the above embodiments, in the process of user creation, the method of assisting user visual arts education operation further includes: detecting second information of the user; the second information includes at least one of the user's blood oxygen and heart rate detected by the robotic arm during the creation process; when the second information does not meet the preset conditions, issuing a warning message, and / or controlling the robotic arm to stop assisting the user's hand; wherein, the warning message is configured to remind the user to stop creating.

[0104] In some embodiments, the warning message may be issued by the reminder module mentioned above, as detailed above, and will not be repeated here.

[0105] In some embodiments, the robot system includes a controller connected to a physiological parameter detection module. When at least one physiological parameter fails to meet a preset condition, the controller can automatically control the robotic arm to pause, thereby suspending the robotic arm's assistance to the user's hand. Alternatively, the controller is connected to a reminder module. When the reminder module generates a reminder message, the controller can simultaneously control the robotic arm to pause, thereby suspending the robotic arm's assistance to the user's hand. Suspending the robotic arm's assistance to the user's hand can prevent health risks caused by over-practice while maintaining the continuity of the user's artistic learning. This integration of health monitoring and art education not only enhances the safety of the creative process but also strengthens the health awareness of elderly users through real-time data feedback. This allows the robot system to truly achieve health protection while providing art education services, addressing the pain point of the lack of health monitoring in existing technologies.

[0106] According to some embodiments of this application, the robot system further includes a voice module. During the user's creative process, the method for assisting the user in visual arts education further includes: detecting second information about the user; the second information includes at least one of the user's blood oxygen and heart rate detected by the robotic arm during the creative process; when the second information does not meet preset conditions, controlling the interactive interface to display a warning message, and / or controlling the voice module to broadcast the warning message, and / or controlling the robotic arm to stop assisting the user's hand; wherein the warning message is configured to remind the user to stop creating.

[0107] The voice module corresponds to the voice prompt unit mentioned above, which will not be elaborated on here.

[0108] It is understandable that by detecting the user's second information before controlling the robotic arm to assist the user's hand movements to obtain an artwork related to the artistic creation task, and by detecting the user's second information during the creation process, if any second information does not meet the preset conditions, the interactive interface can be controlled to display warning information, and / or the voice module can be controlled to broadcast warning information, and / or the robotic arm can be controlled to stop assisting the user's hand, thus enabling real-time health monitoring of the user's entire creative process.

[0109] According to some embodiments of this application, the method for assisting users in visual arts education before user creation further includes: adjusting the display information of the interactive interface based on the results of an ability test to make the display information of the interactive interface suitable for the user; the adjustment includes at least one of font adjustment, layout adjustment, and color adjustment. It is understood that the adjustment of the display information of the interactive interface before user creation is a fixed adjustment, which can avoid interference from subsequent real-time dynamic changes and ensure that the interface remains clear throughout the user's experience.

[0110] In some embodiments of this application, the displayed content of the interactive interface can be adjusted based on the ability test results, and / or based on the environment (e.g., based on the light intensity detected by an environmental perception sensor). For example, the lines (e.g., adjusted to thicker lines), colors (e.g., adjusted to colors with significant contrast), fonts (e.g., adjusted to larger fonts), and layout (e.g., adjusted to the center position) of the displayed content can be adjusted; or, in low light conditions, the display brightness of the interactive interface can be increased. The timing of adjusting the displayed content of the interactive interface can be after obtaining the ability test results, before the user begins the creative operation, or before the artistic creation task is formed.

[0111] It is understandable that adjusting the content displayed on the interactive interface can significantly improve visual readability, reduce user fatigue, and prevent the elderly, people with poor eyesight, and other groups from having to repeatedly adapt to the interactive interface.

[0112] According to some embodiments of this application, please refer to Figure 5 , Figure 5 This is a schematic diagram of the structure of a robot system provided in some embodiments of this application. The robot system 10 includes an interactive interface 101, a robotic arm 102, a voice module 103, a memory 104, and a controller 105.

[0113] The memory 104 is used to store computer programs, and the controller 105 is used to control the interactive interface 101, the robotic arm 102 and the voice module 103 based on the computer programs to implement the user-assisted visual arts education operation method of any of the above embodiments, which will not be described in detail here.

[0114] The robotic arm 102 provided in this application can assist users throughout the entire creative process. Alternatively, the robotic arm 102 provided in this application can automatically provide precise and stable assistance only when it detects tremors or instability in the user's movements. The assistance force can be adjusted according to real-time data from tactile feedback sensors or force feedback sensors. For example, the guiding force can be reduced when the tremor amplitude increases, thereby improving the stability and smoothness of the artistic creation process such as calligraphy or painting.

[0115] Understandably, the approach of only responding when user movements are detected to be jerky or unstable not only reduces the difficulty of creation but also avoids dependency issues caused by excessive intervention and full-process assistance, thus preserving user autonomy.

[0116] The controller 105 involved in this application may be referred to as a CPU (Central Processing Unit), which may be an integrated circuit chip, or a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component.

[0117] The memory 104 used in this application includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), or optical discs.

[0118] In other embodiments, the robot system 10 may also include a capability testing device, an image acquisition device, a force detection module, a speed detection module, a physiological parameter detection module, and a reminder module (indicator light reminder unit), etc. For details, please refer to the above text, which will not be repeated here.

[0119] In other embodiments, the robot system 10 may further include a touch component, which is a handheld input device held by the user for executing operation commands via a touch interface 101. The touch component allows the user to interact with the robot system 10 in a direct and intuitive manner, such as selecting menus, entering text, logging into a user-related account, adjusting the displayed information on the interface 101, or controlling the creation process. The touch component is held by the user for interaction with the interface 101. The touch component may include a haptic feedback sensor or other haptic feedback sensors.

[0120] Understandably, the introduction of touch components can enhance the interactive experience for elderly users. By providing a grip-based touch operation, it can solve voice interaction barriers caused by hearing or vision deterioration and avoid interference from environmental noise on voice recognition. This makes the robot system 10 more adaptable to the physiological characteristics of elderly users, enhancing the accuracy and convenience of interaction, thereby increasing participation and completion rates in art creation and learning, allowing elderly users to focus more on art practice without worrying about the complexity of interaction.

[0121] In some embodiments of this application, the robot system 10 simultaneously supports voice input and touch input functions. There is no priority conflict between the two during interaction, and users can freely choose to use touch or voice input for interaction according to their own needs (such as hearing impairment or environmental noise interference), without the robot system 10 forcibly switching or restricting them. Alternatively, touch input may take priority.

[0122] In other embodiments, the robot system 10 may also include an interactive body, and the interactive interface 101, robotic arm 102, voice module 103, memory 104 and controller 105 may be disposed on the interactive body, so that the robot system 10 can achieve a compact structure.

[0123] The interactive body serves as the physical support structure of the robot system 10, with the controller 105, interactive interface 101, and robotic arm 102 (robotic arm body) all fixedly mounted on it. For example, the interactive body can be designed as a desktop base, with the controller 105 integrated into the internal electronic compartment, the interactive interface 101 such as a large-size, high-contrast touchscreen embedded in the front of the base, and the robotic arm 102 mounted on the top of the base via a rotary joint.

[0124] It is worth noting that the silicone ring at the end of the robotic arm 102 is a flexible wearable component, worn on the user's wrist to integrate health monitoring functions such as a heart rate sensor. However, as the end effector of the robotic arm 102, the silicone ring does not alter the characteristic of the robotic arm itself being mounted on the interactive interface. The interactive interface 101 initializes the font size, color contrast, and interface layout based on the capability test results, and remains static after setting, requiring no real-time adjustments. Health monitoring is directly achieved by the sensor embedded in the silicone ring, eliminating the need for an external wristband or watch. Heart rate data is automatically collected via the silicone ring, without priority conflict with voice input or touch input. The system supports both interaction methods simultaneously, allowing users to choose freely.

[0125] According to some embodiments of this application, please refer to Figure 6 , Figure 6 This is a schematic diagram of the structure of an interactive system provided in some embodiments of this application. The interactive system 100 includes an interactive platform 20 and the robot system 10 described above. The robot system 10 is communicatively connected to the interactive platform 20.

[0126] The robot system 10 is used to: upload artworks to the interactive platform 20; and / or, after uploading artworks to the interactive platform 20, receive feedback information sent by the interactive platform 20.

[0127] When the interactive platform 20 is a client, the art work can be shown to other users who use the client. After the art work is uploaded to the interactive platform 20, the robot system 10 can receive the feedback information sent by the interactive platform 20, and the feedback information can be displayed on the interactive interface 101. It can be understood that the mechanism of storing the art work in the cloud and sharing it socially can promote family community interaction and comments, enhance learning sustainability and social incentive effects, and generally solve the problems of lack of personalization, security and social incentives in the prior art, making users' art learning more convenient, safe and interesting.

[0128] In summary, in an application scenario, the robot system 10 includes a core control module, an aging-friendly interaction module, a creation assistance module, a multi-modal perception module, an AI teaching and motion analysis module, a health monitoring module, and a network interaction module.

[0129] The UI display subunit in the aging-friendly interaction module can fixedly adjust the font size, color contrast, and interface layout according to the results of the vision test and hand flexibility test pre-conducted by the user. For example, the font for users with poor eyesight can be set to 24pt and a high-contrast color scheme can be adopted.

[0130] The end of the robotic arm 102 in the creation assistance module is equipped with a silicone ring that can be sleeved on the user's wrist and integrated with a heart rate sensor for health monitoring; when detecting the user's action jitter, the robotic arm 102 can provide stable assistance and adjust the force in real time according to the feedback of the tactile sensor.

[0131] The AI teaching and motion analysis module can preset the creation process of the target work to be decomposed into multiple steps. For example, the calligraphy character 'yong' is decomposed into dots, horizontal strokes, left-falling strokes, and right-falling strokes, and step-by-step demonstrations are carried out in sequence.

[0132] The health monitoring module automatically collects heart rate and posture data through the sensors embedded in the silicone ring, and triggers a safety prompt and pauses the robotic arm 102 when an abnormality is detected.

[0133] The network interaction module can save the work to the cloud and implement family or community sharing through the interactive platform 20.

[0134] In summary, in an application scenario, based on the above-mentioned interactive system 100 or the robot system 10 of any of the above embodiments, there is the following method:

[0135] S01: Start the system.

[0136] S02: The user logs in.

[0137] Among them, the user login methods include face recognition (involving the above-mentioned image acquisition device) and voiceprint recognition (involving the above-mentioned voice reminder unit or voice module 103).

[0138] S03: Ability Test.

[0139] The ability test here can be a visual and (hand) motor ability test.

[0140] S04: Generate personalized interactive interfaces and course pacing.

[0141] The personalized interactive interface here can automatically adjust based on the visual and (hand) motor ability test results above, such as adjusting font, color contrast, and layout. The course pacing here relates to the guiding information mentioned above.

[0142] S05: AI demonstration of creation steps.

[0143] Here, guidance information can be displayed on the interactive interface 101.

[0144] S06: User creation (robotic arm assistance).

[0145] S07: Real-time motion analysis and error correction.

[0146] In some embodiments, the user's actions during creation can be compared with a standard model, and error correction prompts can be issued when there are inconsistencies.

[0147] S08: Health status monitoring and reminders.

[0148] Here, users' heart rate, body temperature, blood oxygen, etc. can be monitored in real time, and reminder messages can be issued when preset conditions are not met.

[0149] S09: Save and upload your work.

[0150] Here, the artwork created after completion can be saved and uploaded to the interactive platform 20.

[0151] S10: Family or community reviews and interactions.

[0152] Here, after an artwork is uploaded to the interactive platform 20, it can receive comments or interactive messages from family or community. The comments and interactive messages can be directed at the uploaded artwork or other things; there are no restrictions here.

[0153] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for assisting users in visual arts education, characterized in that, Applied to a robotic system, the robotic system including an interactive interface and a robotic arm, at least a portion of the robotic arm being fitted onto a user's hand, the method for assisting the user in visual arts education includes: Based on the user's ability test results, create an art creation task suitable for the user; The interactive interface displays guidance information corresponding to the artistic creation task; the guidance information is used to guide the user to perform creative operations on the interactive interface. During the user's creation process, the robotic arm is controlled to assist the user's hand movements, resulting in an artwork related to the artistic creation task.

2. The method for assisting users in visual arts education according to claim 1, characterized in that, Before controlling the robotic arm to assist the user's hand movements to obtain an artwork related to the artistic creation task during the user's creation process, the method for assisting the user's visual arts education operation further includes: Obtain the user's hand trajectory information; Based on the hand trajectory information, a guidance path corresponding to the artistic creation task and / or the guidance information is generated; During the user's creation process, the robotic arm is controlled to assist the user's hand movements to obtain an artwork related to the artistic creation task, including: During the user's creation process, the robotic arm is controlled to assist the user's hand movements according to the guide path, so as to obtain an artwork related to the artistic creation task on the interactive interface.

3. The method for assisting users in visual arts education according to claim 2, characterized in that, During the process of controlling the robotic arm to assist the user's hand movements, the method for assisting the user's visual arts education includes: Obtain the user's first information; wherein, the first information includes at least one of the force and speed detected on the interactive interface during the creation process; Based on the first information, the assist mode of the robotic arm for the user is adjusted.

4. The method for assisting users in visual arts education according to claim 1, characterized in that, During the user's creative process, the method for assisting the user in visual arts education also includes: During the creation of the artwork, when the current data of the artwork differs from the current guidance data in the guidance information, a prompt message is displayed on the interactive interface; the prompt message is used to remind the user to adjust the artwork according to the current guidance data.

5. The method for assisting users in visual arts education according to claim 1, characterized in that, During the user's creative process, the method for assisting the user in visual arts education also includes: The second information of the user is detected; the second information includes at least one of the user's blood oxygen and heart rate detected by the robotic arm during the creation process. When the second information does not meet the preset conditions, a warning message is issued, and / or the robotic arm is controlled to stop assisting the user's hand; wherein the warning message is configured to remind the user to stop creating.

6. The method for assisting users in visual arts education according to claim 5, characterized in that, The robot system also includes a voice module; The step of issuing a warning message when the second information does not meet the preset conditions includes: When the second information does not meet the preset conditions, the interactive interface is controlled to display warning information, and / or the voice module is controlled to broadcast the warning information.

7. The method for assisting users in visual arts education according to claim 1, characterized in that, Before creating an art creation task suitable for the user based on the user's ability test results, the method for assisting the user in visual arts education further includes: The second information of the user is detected; the second information includes at least one of the user's blood oxygen and heart rate detected by the robotic arm during the creation process. When the second information meets the preset conditions, the step of creating an art creation task suitable for the user is executed based on the user's ability test results.

8. The method for assisting users in visual arts education according to claim 1, characterized in that, The robot system interacts with the interactive platform; After obtaining the artwork related to the artistic creation task, the method for assisting users in visual arts education further includes: Upload the artwork to the interactive platform; and / or After uploading the artwork to the interactive platform, the user receives feedback information from the interactive platform.

9. The method for assisting users in visual arts education according to any one of claims 1-8, characterized in that, Prior to the user's creation, the method for assisting users in visual arts education also includes: Based on the capability test results, the display information of the interactive interface is adjusted to make the display information of the interactive interface suitable for the user; the adjustment includes at least one of font adjustment, layout adjustment and color adjustment.

10. A robot system, characterized in that, include: Interactive interface; robotic arm; Voice module; A memory and a controller, wherein the memory is used to store a computer program, and the controller is used to control the robotic arm, the interactive interface, and the voice module based on the computer program to implement the user-assisted visual arts education operation method according to any one of claims 1-9.

11. An interactive system, characterized in that, include: Interactive platform; The robot system of claim 10, wherein the robot system is communicatively connected to the interactive platform, and the robot system is used for: Upload the artwork to the interactive platform; and / or After uploading the artwork to the interactive platform, the user receives feedback information from the interactive platform.

Images