Multi-mode pet interaction panel direct key-in starting method and multi-mode pet interaction panel direct key-in starting system

Abstract

The invention provides a multi-mode pet interaction panel direct key-in starting method and system, and relates to the technical field of pet interaction.The method comprises the steps that a plurality of specific instructions are preset to serve as shortcut keys for starting a pet interaction panel; directly typing a set shortcut key through the touch screen, and monitoring and identifying whether the input shortcut key is matched with the preset shortcut key in real time to obtain a matching result; according to a matching result, starting a pet interaction panel, and displaying a multi-mode interaction interface comprising a plurality of interaction modes; after the multi-mode interactive interface is displayed, initial frog sound search parameters are set, and the parameters comprise search space and iteration times. According to the method, the pet interaction panel is quickly started through the shortcut key, the multi-mode interaction interface is combined, user preferences are responded in real time, and personalized and immersive interaction experience is provided.

Description

Technical Field

[0001] The present invention relates to the field of pet interaction technology, and in particular to a method and system for directly typing in and starting a multi-modal pet interaction panel. Background Art

[0002] Traditional multimodal pet interaction systems are designed with complex operation interfaces and processes. Users need to go through multiple steps when starting the system or switching interaction modes, which reduces the user's convenience. Especially for pet owners who are not familiar with electronic devices, this complexity may become a barrier for them to use such systems.

[0003] In terms of command input, traditional systems lack intuitive and efficient input methods. Users may need to enter commands through cumbersome menu navigation, specific key combinations or complex voice commands, which not only increases the difficulty of operation, but may also lead to input errors, affecting the smoothness and accuracy of interaction. Traditional systems may still have problems with low accuracy or slow response when recognizing user input (such as voice, gestures, facial expressions, etc.). This may cause the system to be unable to accurately understand the user's intentions, or to be unable to provide feedback in a timely manner, thereby affecting the user's interactive experience.

[0004] In addition, traditional multimodal pet interaction systems lack sufficient personalization and customization options. Users may not be able to adjust the interactive content, interface style, or feedback method according to their preferences or the characteristics of their pets, which makes the interactive experience of the system relatively single and difficult to meet the needs of different users. Traditional systems may only support specific devices or platforms, which limits users' device selection and usage scenarios. At the same time, as technology continues to develop, it may be difficult to adapt to new recognition technologies or interaction methods, resulting in deficiencies in scalability and sustainability. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a multimodal pet interactive panel direct typing start method and system, which realizes fast and accurate command input and provides a rich and personalized interactive experience through a multimodal feedback mechanism.

[0006] In order to solve the above technical problems, the technical solution of the present invention is as follows:

[0007] In a first aspect, a multimodal pet interactive panel direct typing start method, the method comprising:

[0008] Pre-set multiple specific commands as shortcut keys to start the pet interaction panel;

[0009] Directly input the set shortcut key through the touch screen, and monitor and identify in real time whether the input shortcut key matches the preset shortcut key to obtain the matching result;

[0010] According to the matching results, the pet interaction panel is started and a multimodal interaction interface including multiple interaction modes is displayed;

[0011] After the multimodal interactive interface is displayed, set the parameters for the initial frog sound search, including the search space and the number of iterations;

[0012] The touch screen monitors and analyzes the user's interactive operations in real time, including touch and voice input, and performs iterative searches based on the user's interactive operations, simulating the frog's call communication process, constantly listening to and learning the user's preferences, in order to identify and determine the user's interactive mode, and obtain recognition results;

[0013] Based on the recognition results, the content of the multimodal interactive interface is updated in real time to reflect the user's preferred interaction method, and the interaction results are fed back to the user through visual, auditory or tactile sensory channels.

[0014] Furthermore, the set shortcut key is directly entered through the touch screen, and real-time monitoring and identification are performed to determine whether the input shortcut key matches the preset shortcut key, and a matching result is obtained, including:

[0015] Send instructions to the touch screen driver to activate the continuous monitoring function of the user's touch input, including the location of the touch point, the type of touch event and the duration of the touch;

[0016] When the touch screen driver captures a touch event, it receives the corresponding event data and determines whether the touch point is located in the preset shortcut key input area. If the touch event occurs in the shortcut key input area, the touch screen driver reads the character sequence input by the user;

[0017] Start the regular expression matching process, traverse the preset shortcut key list and match it with the character sequence entered by the user. If the match is successful, execute the operation associated with the matching shortcut key, including starting the pet interaction panel; if there is no matching result after traversing the list, maintain the monitoring state and wait for the user to re-enter to obtain the matching result.

[0018] Furthermore, according to the matching results, the pet interaction panel is started, and a multimodal interaction interface containing multiple interaction modes is displayed, including:

[0019] Based on the matching results, a start instruction is sent to the pet interactive panel system, including the user identity and interaction mode preference;

[0020] The pet interactive panel system receives the startup command and extracts the resources of the multimodal interactive interface, including images, audio, video files and interactive logic scripts;

[0021] After the resource extraction is completed, the multimodal interactive interface is initialized, including setting the interface layout, color, font visual elements and configuring audio output and touch input interaction methods;

[0022] After initializing the multimodal interactive interface, the pet interactive panel system displays the multimodal interactive interface to the user through the touch screen.

[0023] Furthermore, the multimodal interactive interface allows users to interact with pets in multiple ways such as touch, voice, and vision.

[0024] Furthermore, the user's interactive operations, including touch and voice input, are monitored and analyzed in real time through the touch screen, and iterative searches are performed according to the user's interactive operations, simulating the frog's call communication process, continuously listening to and learning the user's preferences, so as to identify and determine the user's interactive mode, and obtain recognition results, including:

[0025] Start the real-time monitoring function and voice recognition function of the touch screen. When a touch event or voice input is detected, analyze the user's operation intention, including clicking a button, sliding a slider, and speaking a specific command;

[0026] Iterative search is performed based on the detected touch events and voice input to simulate the frog's call communication process. During the iteration process, the fitness of the current interaction mode is continuously evaluated to obtain the fitness evaluation result;

[0027] Based on the fitness evaluation results, learn and identify the user's preferred interaction method. When the preset number of iterations is reached, output the recognition results, including the user's preferred interaction method and commonly used operation instructions.

[0028] Furthermore, the calculation formula of the fitness of the current interaction mode is:

[0029]

[0030] Among them, F represents the fitness of the current interaction mode; α1, α2, α3, α4 represent weight coefficients; β1, β2, β3, β4, β5 represent adjustment coefficients; T r Represents reaction time; T m represents the maximum acceptable reaction time; δ1, δ2, δ3, δ4, δ5 represent the offset; N i Indicates the number of interactions; N e Indicates the preset number of interactions; T u Indicates the usage time within the evaluation period; T h Indicates the usage time threshold; P p Indicates the proportion of positive feedback; N f represents the total number of feedbacks given during the evaluation period; N n Indicates the minimum feedback quantity.

[0031] Furthermore, based on the recognition results, the content of the multimodal interactive interface is updated in real time to reflect the user's preferred interaction method, and the interaction results are fed back to the user through visual, auditory or tactile sensory channels, including:

[0032] Generate content that matches the user's preferences based on the recognition results, including changing the arrangement of buttons, displaying different images or animations, and updating the content that matches the user's preferences on the multimodal interactive interface in real time;

[0033] Displaying content that matches user preferences, including images and animated visual elements, updated in real time through a touch screen to obtain recognition results and interface content;

[0034] Based on the recognition results and interface content, audio feedback corresponding to the user interaction is generated, including cheers, encouragement, and pet calls.

[0035] In the second aspect, a multimodal pet interactive panel direct typing start system includes: an input recognition module for receiving and recognizing a user's direct typing content, including text, symbols, numbers, and user input captured by a touch screen or microphone, including gestures, voice, or facial expressions, to obtain a recognition result;

[0036] A preference analysis module is used to analyze the recognition results and determine the user's preferences and intentions, including the user's preferred interaction method, pet type, and interaction scenario, to obtain analysis results;

[0037] An interface update module, which is used to update the content of the multimodal interactive interface in real time according to the analysis results of the preference analysis module, including changing the arrangement of buttons, displaying images or animations that match user preferences, and adjusting the color and layout of the interface;

[0038] A feedback generation module, which is used to generate corresponding audio, visual or tactile feedback according to the interface content, including cheers, encouragement, pet calls, dynamic images or vibration feedback;

[0039] An output display module, including a touch screen, a speaker, and a vibration motor, is used to display updated interface content to the user and play or provide generated feedback;

[0040] The control logic module is used to coordinate the work of each module, respond to user input, and provide feedback.

[0041] According to a third aspect, a computing device includes:

[0042] one or more processors;

[0043] The storage device is used to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method described.

[0044] In a fourth aspect, a computer-readable storage medium stores a program, and when the program is executed by a processor, the method described is implemented.

[0045] The above solution of the present invention includes at least the following beneficial effects:

[0046] By presetting multiple specific commands as startup shortcuts, users can quickly start the pet interactive panel by simply typing these shortcuts on the touch screen, without going through complicated operation processes or navigating menus, which improves the convenience of use. The iterative search mechanism simulates the frog's call communication process. By real-time monitoring and parsing the user's interactive operations (including touch and voice input), the system can continuously learn and adapt to the user's preferences, thereby interacting with the user in a more natural and personalized way, enhancing the realism and fun of the interactive experience.

[0047] By real-time monitoring and identifying the shortcut keys input by users, and subsequent iterative searches based on user interaction operations, the system can accurately identify the user's intentions and needs and respond quickly. This mechanism not only improves the accuracy of recognition, but also shortens the system's response time, making the interaction smoother and more immediate. Based on the user's interactive operations and preferences, the system can update the content of the multimodal interactive interface in real time to reflect the user's preferred interaction method. This personalized customization not only meets the needs of different users, but also increases the diversity and fun of interaction.

[0048] The system uses multiple sensory channels such as vision, hearing or touch to feedback the interaction results to the user, such as displaying dynamic images, playing sound effects or providing vibration feedback. This rich sensory feedback mechanism allows users to be more immersed in the interaction with their pets and enjoy a more realistic and vivid interactive experience. The touch screen input and iterative search mechanism have wide applicability and scalability. With the continuous development of technology, the system can easily integrate new recognition technologies or interaction methods, while supporting multiple devices and platforms to meet the diverse usage needs of users. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is a flowchart of a method for directly typing in and starting a multimodal pet interactive panel provided by an embodiment of the present invention.

[0050] Figure 2 It is a schematic diagram of a direct typing start system of a multimodal pet interactive panel provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0051] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0052] like Figure 1 As shown, an embodiment of the present invention proposes a multi-modal pet interactive panel direct typing start method, the method comprising the following steps:

[0053] Step 11, presetting a plurality of specific commands as shortcut keys for starting the pet interactive panel;

[0054] Step 12, directly inputting the set shortcut key through the touch screen, and monitoring and identifying in real time whether the input shortcut key matches the preset shortcut key to obtain a matching result;

[0055] Step 13, based on the matching result, start the pet interaction panel and display a multimodal interaction interface including multiple interaction modes;

[0056] Step 14, after the multimodal interactive interface is displayed, the parameters of the initial frog sound search are set, including the search space and the number of iterations;

[0057] Step 15, monitoring and analyzing the user's interactive operations in real time through the touch screen, including touch and voice input, and performing iterative search according to the user's interactive operations, simulating the frog's call communication process, continuously listening to and learning the user's preferences, so as to identify and determine the user's interactive mode, and obtain the recognition result;

[0058] Step 16, based on the recognition results, update the content of the multimodal interactive interface in real time to reflect the user's preferred interaction method, and feedback the interaction results to the user through visual, auditory or tactile sensory channels.

[0059] In the embodiment of the present invention, by presetting shortcut keys, users can quickly and directly start the pet interactive panel without going through a tedious startup process or looking for a startup button, which improves the convenience and efficiency of use. This design enables pet owners to quickly interact with their pets whenever they need, enhancing the immediacy of the interactive experience.

[0060] Step 12: The direct input method of the touch screen is intuitive and easy to operate. Users can enter shortcut keys by simply touching the screen. The real-time monitoring and recognition mechanism ensures that the system can respond quickly to user input and accurately determine whether it matches the preset shortcut keys, thereby quickly starting the pet interactive panel, reducing user waiting time and improving the system's response speed.

[0061] Step 13, quickly start the pet interaction panel according to the matching results, and display a variety of interaction modes, providing users with a wealth of choices. The design of the multimodal interaction interface allows users to interact with pets through multiple sensory channels such as vision and hearing, enhancing the diversity and immersion of the interactive experience.

[0062] Step 14, by setting the parameters of the initial frog sound search, the system can conduct targeted searches to more efficiently identify the user's preferences and interaction methods. Reasonable settings of the search space and number of iterations not only ensure the accuracy of the search, but also avoid unnecessary waste of computing resources and improve the overall performance of the system.

[0063] Step 15, real-time monitoring and analysis of user interaction operations, so that the system can instantly obtain user feedback and needs. Through iterative search and simulation of frog calls, the system can continuously learn and adapt to user preferences, thereby more accurately identifying and determining the user's interaction method. This mechanism not only improves the accuracy of recognition, but also enhances the system's sensitivity and adaptability to user needs.

[0064] Step 16: Update the content of the multimodal interactive interface in real time according to the recognition results, so that the interface is more in line with the user's preferences and needs. Feedback of the interaction results to the user through multiple sensory channels such as vision, hearing or touch not only enhances the richness and realism of the interactive experience, but also enables the user to more intuitively feel the interaction effect with the pet, thereby further improving the user's satisfaction and loyalty.

[0065] In a preferred embodiment of the present invention, the above step 11, presetting a plurality of specific instructions as shortcut keys for starting the pet interactive panel, may include:

[0066] In an embodiment of the present invention, a specific set of instructions to be used as shortcut keys for starting the pet interactive panel is determined. These instructions can be a combination of keys on the keyboard (such as Ctrl+Alt+P) or a specific button combination on the handle (such as LB+RB). A UI interface for the pet interactive panel is developed, including the pet's image, status display, interactive options, etc. The display logic and interaction logic of the panel are determined, such as how to respond to the player's input, how to update the pet's status, etc. The pet interactive panel is integrated into the game system or application supported by the game controller to ensure that the panel can display and interact normally.

[0067] In the firmware of the game controller, a monitoring mechanism is set to capture input signals from the controller. For keyboard key combinations, monitoring is required at the application level. When the input signal is captured, it is matched with the predefined shortcut key instruction set. If the match is successful, the startup logic of the pet interactive panel is triggered. Once a valid shortcut key instruction is identified, the pet interactive panel is immediately started. Ensure that the display and interaction logic of the panel can operate normally, such as loading pet images, displaying status information, responding to player operations, etc. In the settings menu of the game controller, a configuration interface for shortcut key instructions is provided. Allow players to customize or modify shortcut key instructions to meet different gaming habits and needs. Save the shortcut key instructions configured by the player to the memory of the controller or the configuration file of the application. Ensure that the configuration information can be restored when the controller is restarted or the application is reopened. When configuring shortcut key instructions, conflict detection is performed to avoid conflicts between different shortcut key instructions. If a conflict is detected, prompt the player to reconfigure or select another instruction.

[0068] In a preferred embodiment of the present invention, the above step 12, directly inputting the set shortcut key through the touch screen, and real-time monitoring and identifying whether the input shortcut key matches the preset shortcut key to obtain the matching result, may include:

[0069] Step 121, sending an instruction to the touch screen driver to activate the continuous monitoring function of the user's touch input, the monitoring content includes the position of the touch point, the type of touch event and the duration of the touch;

[0070] Step 122, when the touch screen driver captures a touch event, it receives corresponding event data and determines whether the touch point is located in a preset shortcut key input area. If the touch event occurs in the shortcut key input area, the touch screen driver reads the character sequence input by the user;

[0071] Step 123, start the regular expression matching process, traverse the preset shortcut key list and match it with the character sequence input by the user. If the match is successful, execute the operation associated with the matching shortcut key, including starting the pet interaction panel; if there is no matching result after traversing the list, maintain the monitoring state and wait for the user to re-enter to obtain the matching result.

[0072] In an embodiment of the present invention, a touch screen driver is loaded to ensure that it communicates correctly with the hardware. Driver parameters such as touch sensitivity, sampling rate, etc. are configured. Instructions are sent to the touch screen driver through the operating system to enable continuous monitoring mode. The instructions contain specific information that needs to be monitored, such as the location of the touch point (x, y coordinates), the type of touch event (such as press, slide, lift), and the duration of the touch. A callback function is defined that will receive the touch event data captured by the touch screen driver. Ensure that the signature of the callback function is consistent with that described in the API documentation so that the driver can call it correctly. Register a pointer or reference to the callback function with the touch screen driver, ensure that the registration is successful, and check any possible errors or status codes.

[0073] In the callback function, perform only necessary operations, such as updating the user interface, recording touch data, or triggering specific events. Avoid performing time-consuming or complex tasks in the callback function. Move these tasks to asynchronous threads or background tasks. If the callback function needs to process a large amount of touch data, consider using efficient data structures to store and process this data, such as queues, hash tables, or arrays. Implement error handling logic in the callback function to deal with possible exceptions, such as invalid touch event data or insufficient resources. Record error logs or send error notifications to diagnose and solve problems in a timely manner. Fully test and debug the callback function to ensure that it works correctly in various situations and does not introduce any delays or loss issues.

[0074] Step 122, when the touch screen driver captures a touch event, the corresponding event data is received through the callback function. The data should contain information such as the location of the touch point, the event type and the timestamp. Based on the location information of the touch point, it is determined whether the touch event occurs in the preset shortcut key input area. This involves comparing the coordinates of the touch point with the boundaries of the shortcut key input area. If the touch event occurs in the shortcut key input area, the character sequence input by the user is read through the touch screen driver. This involves converting the movement trajectory of the touch point into character input, or recognizing a specific touch gesture as input.

[0075] Step 123, load a preset shortcut key list, each shortcut key is represented as a specific character sequence or pattern. Start the regular expression matching process, and match the character sequence entered by the user with each shortcut key in the shortcut key list. Compare the character sequence entered by the user with each shortcut key in the shortcut key list in turn. If a match is found, jump out of the loop; if no match is found after traversing the list, remain in the monitoring state. If the match is successful, perform the corresponding operation according to the matched shortcut key. For the shortcut key that starts the pet interactive panel, trigger the display and interaction logic of the panel. If the match fails, wait for the user to re-enter and repeat the above matching process.

[0076] Assume that the user presets a shortcut key "P+I+N" (short for "Pet Interaction Now") to start the pet interaction panel. The user touches the shortcut key input area on the screen and enters "P", "I", and "N" in sequence. The touch screen driver captures these touch events and reads the character sequence "P+I+N" entered by the user. The regular expression matching process is started to match "P+I+N" with the preset shortcut key list. After finding a match, the system performs the operation associated with the "P+I+N" shortcut key, which is to start the pet interaction panel.

[0077] Users can start the pet interactive panel by directly touching the screen to enter shortcut keys, without the need for additional physical buttons or complex operation processes. The function of real-time monitoring and identification of input shortcut keys allows users to get instant feedback, enhancing the smoothness and intuitiveness of the interaction. Users can customize shortcut keys according to their preferences and habits, which improves the flexibility and personalization of the system. The system can continuously monitor and identify shortcut keys entered by users, allowing users to start the pet interactive panel at any time without interrupting the current game or application. Through the regular expression matching process, the system can efficiently process shortcut keys entered by users and quickly perform associated operations. This reduces the system's response time and processing delay, and improves overall operating efficiency.

[0078] In a preferred embodiment of the present invention, the above step 13, based on the matching result, starts the pet interaction panel and displays a multimodal interaction interface including multiple interaction modes, which may include:

[0079] Step 131, based on the matching result, sending a start instruction to the pet interactive panel system, including the user identity and interactive mode preference;

[0080] Step 132, the pet interactive panel system receives the start instruction and extracts the resources of the multimodal interactive interface, including images, audio, video files and interactive logic scripts;

[0081] Step 133, after the resource extraction is completed, the multimodal interactive interface is initialized, including setting the layout, color, font visual elements of the interface and configuring the audio output and touch input interaction mode;

[0082] Step 134, after initializing the multimodal interactive interface, the pet interactive panel system displays the multimodal interactive interface to the user through the touch screen.

[0083] In an embodiment of the present invention, the user's identity information and interaction mode preference are obtained from the user identification system or the previous interaction process. The matching result includes the user's ID, name, pet type preference, and interaction mode preference (such as games, education, feeding, etc.). According to the matching result, a startup instruction data packet containing the user's identity and interaction mode preference is constructed. The data packet includes the user ID, interaction mode identifier, and other necessary parameters (such as pet type, interaction level, etc.). The startup instruction data packet is sent to the pet interaction panel system through a network communication protocol (such as TCP / IP, HTTP, etc.).

[0084] Step 132, the pet interactive panel system listens for a startup command from the user identification system. When the startup command is received, the data packet is parsed to obtain the user identity and the interaction mode preference. According to the user identity and the interaction mode preference, the required multimodal interactive interface resources are located in the resource library. The resources may include image files (such as background images, pet icons), audio files (such as background music, pet calls), video files (such as animations, tutorial videos), and interactive logic scripts.

[0085] Step 133, according to the interaction mode preference and the preset layout template, set the overall layout of the interface, including the position and size of each element. Set the color scheme, font style and size and other visual elements of the interface to ensure the aesthetics and readability of the interface. Load and display image resources, such as background images, pet icons, etc. Initialize audio output devices, such as speakers or headphones. Load and prepare to play audio resources, such as background music, pet calls, etc. According to the interactive logic script, configure the input interaction method of the touch screen, such as touch point recognition, sliding operation response, etc.

[0086] Step 134, render the initialized multimodal interactive interface onto the touch screen. Ensure that the rendering of the interface is smooth without lag or flickering. Monitor the user's touch input in real time and process the input event according to the interactive logic script. Update the interface state to reflect the user's operation results, such as moving the pet icon, playing animation, etc. Provide feedback to the user through audio, visual or tactile means, such as playing sound effects, displaying prompt information, etc.

[0087] Assume that user A likes to play interactive games with virtual pets. The previous matching results show that user A prefers "dog" as the pet type and likes the "catch the frisbee" interactive mode. Send a startup command to the pet interactive panel, including user A's ID and the identifier of the "catch the frisbee" mode. After receiving the command, the pet interactive panel system extracts the "dog" image, the "catch the frisbee" background image, the dog barking audio, and the interactive logic script of the frisbee catching game from the resource library. Initialize the interface, set the green grass as the background, display a cute dog icon, and configure the speaker to play the dog barking and background music. At the same time, set the sliding operation of the touch screen to control the dog to move and catch the frisbee. After the interface rendering is completed, user A sees the dog and the frisbee on the touch screen, and controls the dog to move and catch the frisbee by sliding the screen. Every time the frisbee is caught, the system will play cheerful music and the dog's barking as feedback.

[0088] Through the multimodal interactive interface, users can interact with their pets more intuitively and enjoy a more immersive experience. Touch input and real-time feedback mechanisms allow users to interact with their pets more naturally, increasing the fun and realism of the interaction. Based on the user's identity and preferences, the system can automatically adjust the interactive interface and resources to meet the user's personalized needs. By configuring and loading different resources, the system can easily support multiple interaction modes and pet types, improving the flexibility and scalability of the system.

[0089] In another preferred embodiment of the present invention, the multimodal interactive interface allows the user to interact with the pet in multiple ways such as touch, voice, and vision, which may include:

[0090] In an embodiment of the present invention, the pet interaction system is started and the necessary system components and services are loaded. The images, audio, video files and interactive logic scripts required for the multimodal interactive interface are loaded from the resource library. These resources include different expressions and action images of pets, audio files such as pets' calls and background music, as well as animated videos and logic processing scripts of interactive games. The user's identity is identified through facial recognition, fingerprint recognition or other identity authentication methods. According to the user's historical data or preset configuration, the user's interaction mode preferences, such as favorite pet types, interactive games, etc., are obtained.

[0091] According to the user's preferences and the preset layout template, set the overall layout of the multimodal interactive interface, including the location and size of the pet display area, interactive game area, control buttons, etc. Set the color scheme, font style and size of the interface, and load and display visual elements such as the pet's initial image and background image. Initialize audio output devices, such as speakers or headphones, and prepare to play audio resources such as pet calls and background music. Configure speech recognition and speech synthesis services so that users can interact with pets through voice.

[0092] The user interacts with the pet through the touch screen, such as clicking the pet image to trigger the pet's response, dragging the pet to move, or sliding in the interactive game area. The system updates the interface status in real time based on the user's touch input, such as changing the pet's expression, action, or updating the game progress, and enhances the interactive experience through tactile feedback (such as vibration). The user speaks instructions or talks to the pet through the microphone, and the system uses speech recognition technology to convert the user's voice into text. Based on the recognized text content, the user's intention is judged and the corresponding interactive logic is called for processing. Using speech synthesis technology, the pet's response or the system's prompt information is converted into voice output and played to the user through speakers or headphones. According to the interactive logic, the system updates the pet's image and animation in real time, such as the pet's expression changes, action execution, etc., to provide a visual interactive experience. During the interactive process, the system can add visual effects, such as flashing, gradients, zooming, etc., to enhance the attractiveness and interactive effect of the interface. Determine whether the interactive time reaches the preset end condition. Detect whether the user has performed an operation to end the interaction, such as clicking the exit button or speaking an end instruction. Release the resources occupied by the multimodal interactive interface, such as images, audio, video files and memory. Close the multimodal interactive interface and return to the system's main interface.

[0093] In a preferred embodiment of the present invention, in the above step 14, after the multimodal interactive interface is displayed, the parameters of the initial frog sound search are set, including the search space and the number of iterations, which may include:

[0094] In the embodiment of the present invention, the search space may involve the following aspects:

[0095] Visual parameters: such as the color range, brightness, contrast, etc. of the interface.

[0096] Audio parameters: such as the volume and pitch of background music, variations of pet calls, etc.

[0097] Interaction logic parameters: such as the sensitivity of pet reactions, game difficulty, switching thresholds for interaction modes, etc.

[0098] For each parameter to be optimized, its minimum and maximum values ​​are determined, thus defining a continuous search space. The number of iterations determines how many iterations the frog search algorithm will run to find the final solution.

[0099] In a preferred embodiment of the present invention, the above step 15, monitoring and analyzing the user's interactive operations in real time through the touch screen, including touch and voice input, and performing iterative search according to the user's interactive operations, simulating the frog's call communication process, constantly listening to and learning the user's preferences, to identify and determine the user's interactive mode, to obtain the recognition result, may include:

[0100] Step 151, start the real-time monitoring function and voice recognition function of the touch screen, and when a touch event or voice input is detected, analyze the user's operation intention, including clicking a button, sliding a slider, and speaking a specific command;

[0101] Step 152, performing an iterative search based on the detected touch events and voice input to simulate the frog's call communication process, and in the iterative process, continuously evaluating the fitness of the current interaction mode to obtain a fitness evaluation result;

[0102] Step 153, based on the fitness evaluation result, learn and identify the user's preferred interaction method, and when a preset number of iterations is reached, output the identification result, including the user's preferred interaction method and commonly used operation instructions.

[0103] In an embodiment of the present invention, the touch screen monitoring service and the speech recognition service are initialized. The touch screen monitoring service is responsible for capturing the user's touch events, such as clicks, slides, etc.; the speech recognition service is responsible for converting the user's voice input into text. Configure the sensitivity, response speed and other parameters of the touch screen monitoring, as well as the language model, recognition accuracy and other parameters of the speech recognition to ensure that the user's interactive operations can be accurately captured and analyzed. When the touch screen detects a touch event or the speech recognition service recognizes the voice input, the event processing function is triggered. This function is responsible for parsing the user's operation intention, such as determining which button is clicked, which slider is slid, or recognizing a specific instruction spoken by the user.

[0104] Step 152, define a search space containing multiple possible interaction modes, such as different button layouts, slider ranges, instruction sets, etc. These interaction modes can be regarded as different variations of "frog calls". Set the initial parameters of the iterative search, such as the number of iterations, fitness evaluation function, etc. The fitness evaluation function is used to evaluate the degree of match between the current interaction mode and the user's preference. In each iteration, select an interaction mode from the search space as the current mode. According to the current mode, simulate the frog's call communication process, that is, display interface elements, respond to touch events and voice input according to this mode. Collect user interaction data in this mode, such as operation frequency, dwell time, satisfaction feedback, etc. Use the fitness evaluation function to evaluate the current mode and obtain the fitness evaluation result. According to the fitness evaluation result, update the search space, eliminate the mode with lower fitness, and optimize the mode with higher fitness.

[0105] Step 153, during the iterative search process, continuously accumulate user interaction data, and use machine learning algorithms to analyze and model these data to learn user preferences. When the preset number of iterations is reached or the fitness evaluation result reaches a certain threshold, stop the iterative search. At this point, the system has learned the user's preferences and can identify the user's preferred interaction methods and commonly used operation instructions. The recognition results are displayed to the user in a visual way, such as displaying the user's preferred button layout, slider range, etc. At the same time, the recognition results can also be used to optimize the system's interactive design and enhance the user experience.

[0106] Suppose that when a user uses a multimodal interactive interface, he often triggers a function by touching a specific area on the screen, and often speaks a specific voice command to perform an operation. During the iterative search process, the system will gradually learn these preferences and give priority to displaying the user's preferred button layout and responding to the user's preferred voice commands in the interaction. For example, the function buttons that users often use will be placed in a more prominent position on the screen, and respond quickly when the user speaks a specific command.

[0107] By monitoring and analyzing user interactions in real time, the system can more accurately understand user needs and preferences, thereby providing a more personalized and efficient interactive experience. By iteratively searching and learning user preferences, the system can continuously adapt to user changes and needs and maintain unimpeded interaction with users. By identifying user preferred interaction methods and operation instructions, the system can allocate resources more reasonably, such as prioritizing the loading of user-commonly used functional modules to reduce unnecessary resource consumption.

[0108] In a preferred embodiment of the present invention, the calculation formula of the fitness of the current interaction mode is:

[0109]

[0110] Among them, F represents the fitness of the current interaction mode; α1, α2, α3, α4 represent weight coefficients; β1, β2, β3, β4, β5 represent adjustment coefficients; T r Represents reaction time; T m represents the maximum acceptable reaction time; δ1, δ2, δ3, δ4, δ5 represent the offset; N i Indicates the number of interactions; N e Indicates the preset number of interactions; T u Indicates the usage time within the evaluation period; T h Indicates the usage time threshold; P p Indicates the proportion of positive feedback; N f represents the total number of feedbacks given during the evaluation period; N n Indicates the minimum feedback quantity.

[0111] In the embodiment of the present invention, the weight coefficients (α1, α2, α3, α4), adjustment coefficients (β1, β2, β3, β4, β5), offsets (δ1, δ2, δ3, δ4, δ5), and maximum acceptable reaction time (T m ), preset number of interactions (N e ), usage time threshold (T h ) and the minimum number of feedbacks (N n ) and other parameters. Check the loaded parameters to ensure that they are within a reasonable range to avoid calculation errors or anomalies. Initialization reaction time (T r ), number of interactions (N i ), usage time within the evaluation period (T u ), positive feedback ratio (P p ) and the total number of feedback given during the evaluation period (N f ) and other variables for subsequent data collection and calculation.

[0112] The touch screen and voice recognition module monitor the user's interactive operations in real time, including touch events and voice input. Record the reaction time (T r ), and the cumulative number of interactions (N i At the same time, record the user's usage time during the evaluation period (T u ) and the total number of feedback given (N f ), and the number of positive feedbacks, used to calculate the positive feedback ratio (P p ).

[0113] Calculate the reaction time component

[0114] Calculating the number of interactions

[0115] Calculate the usage time component

[0116] Calculate feedback components

[0117] Add up all the components to get the fitness F of the current interaction mode.

[0118] By calculating fitness in real time and adjusting interaction strategies, the system can more accurately meet user needs and preferences and provide a more personalized and smooth interaction experience. The fitness calculation formula takes into account multiple factors, such as reaction time, number of interactions, usage time, feedback, etc., so that the system can more comprehensively and accurately evaluate the pros and cons of interaction modes, and make adjustments and optimizations accordingly to enhance the adaptability of the system. Through fitness evaluation, users' commonly used interaction methods and operation instructions can be identified, so as to prioritize resource allocation, such as loading speed, processing priority, etc., to improve the system's operating efficiency and response speed.

[0119] In a preferred embodiment of the present invention, the above step 16, based on the recognition result, updates the content of the multimodal interactive interface in real time to reflect the user's preferred interaction mode, and feeds back the interaction result to the user through visual, auditory or tactile sensory channels, which may include:

[0120] Step 161, generating content matching the user's preferences based on the recognition results, including changing the arrangement of buttons, displaying different images or animations, and updating the content matching the user's preferences on the multimodal interactive interface in real time;

[0121] Step 162, displaying content matching the user's preferences updated in real time through the touch screen, including images and animation visual elements, to obtain recognition results and interface content;

[0122] Step 163, generating audio feedback corresponding to the user interaction according to the recognition result and the interface content, including cheers, encouragement, and pet calls.

[0123] In the embodiment of the present invention, the latest user preference data is obtained from the user behavior analysis module, including commonly used interaction methods, preferred content types (such as images, animations), favorite colors, layouts, etc. According to the user preference data, the content library is searched for matching content, such as a specific button arrangement, image or animation material. The graphics processing engine is used to generate interface elements that match the user's preferences based on the matched content, such as adjusting the size, color, and arrangement of buttons, or loading images and animations preferred by the user.

[0124] The generated interface elements are updated in real time to the corresponding positions of the multimodal interactive interface to ensure that users can see content that suits their preferences every time they interact.

[0125] Step 162, render the updated interface elements into the display buffer of the touch screen to ensure smooth display of images and animations. Monitor the touch screen's touch events. When the user touches the screen, determine the user's intention and trigger the corresponding interaction logic based on the touch position and content layout. Display the real-time updated interface content through the touch screen, including user-triggered animation effects, image changes, etc., so that the user can intuitively see the interaction results.

[0126] Step 163, prepare a variety of audio materials in advance, such as cheers, encouragements, pet calls, etc., corresponding to different interaction results and situations. According to the recognition results and interface content, determine the user's current interaction situation and results, and select audio materials that match them from the audio library. Use the audio playback engine to play the selected audio materials, and transmit audio feedback to the user through speakers or headphones to enhance the immersion and fun of the interaction.

[0127] Suppose the user prefers pet-themed interactive interfaces and prefers to hear the pet's barking when clicking a button. After identifying the user's preference, pet-themed images and animations are loaded from the content library, such as puppy icons, bone buttons, etc., and the buttons are arranged in a way that the user usually uses. When the user clicks the bone button on the touch screen, the interface will display an animation effect of a puppy eating a bone, and the button arrangement will be adjusted according to the user's habits. The audio feedback of the puppy's barking is played to let the user feel the fun of interacting with the pet.

[0128] By updating content that matches user preferences in real time and providing visual, auditory and other multi-sensory feedback, users can experience a more personalized and immersive experience during the interaction. Interactive interfaces and feedback mechanisms that meet user preferences can attract users to use the system more frequently, improving user satisfaction and loyalty. Adjusting the interface layout and button arrangement according to user preferences allows users to find and trigger the desired interactive functions more quickly, improving interaction efficiency. Through audio feedback such as pet calls, the emotional communication between the system and the user is enhanced, allowing users to have more emotional dependence and identification with the system. The system can be adjusted and optimized in real time according to user preferences, providing a foundation and support for future personalized customization services.

[0129] like Figure 2 As shown, the embodiment of the present invention also provides a multi-modal pet interactive panel direct input start system, including:

[0130] An input recognition module is used to receive and recognize the user's direct input content, including text, symbols, numbers, and user input captured by a touch screen or microphone, including gestures, voice or facial expressions, to obtain a recognition result;

[0131] A preference analysis module is used to analyze the recognition results and determine the user's preferences and intentions, including the user's preferred interaction method, pet type, and interaction scenario, to obtain analysis results;

[0132] An interface update module, which is used to update the content of the multimodal interactive interface in real time according to the analysis results of the preference analysis module, including changing the arrangement of buttons, displaying images or animations that match user preferences, and adjusting the color and layout of the interface;

[0133] A feedback generation module, which is used to generate corresponding audio, visual or tactile feedback according to the interface content, including cheers, encouragement, pet calls, dynamic images or vibration feedback;

[0134] An output display module, including a touch screen, a speaker, and a vibration motor, is used to display updated interface content to the user and play or provide generated feedback;

[0135] Control logic module, used to coordinate the work of each module, respond to user input, and provide feedback

[0136] It should be noted that the system is a system corresponding to the above method, and all implementation methods in the above method embodiment are applicable to this embodiment and can achieve the same technical effect.

[0137] The embodiment of the present invention further provides a computing device, comprising: a processor, a memory storing a computer program, wherein when the computer program is executed by the processor, the method described above is executed. All implementations in the above method embodiment are applicable to this embodiment and can achieve the same technical effect.

[0138] The embodiment of the present invention also provides a computer-readable storage medium storing instructions, which, when executed on a computer, enable the computer to execute the method described above. All implementations in the above method embodiment are applicable to this embodiment and can achieve the same technical effect.

[0139] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A multi-modal pet interactive panel direct input start method, characterized in that: The method comprises: Pre-set multiple specific commands as shortcut keys to start the pet interaction panel; Directly input the set shortcut key through the touch screen, and monitor and identify in real time whether the input shortcut key matches the preset shortcut key to obtain the matching result; According to the matching results, the pet interaction panel is started and a multimodal interaction interface including multiple interaction modes is displayed; After the multimodal interactive interface is displayed, set the parameters for the initial frog sound search, including the search space and the number of iterations; The touch screen monitors and analyzes the user's interactive operations in real time, including touch and voice input, and performs iterative searches based on the user's interactive operations, simulating the frog's call communication process, constantly listening to and learning the user's preferences, in order to identify and determine the user's interactive mode, and obtain recognition results; Based on the recognition results, the content of the multimodal interactive interface is updated in real time to reflect the user's preferred interaction method, and the interaction results are fed back to the user through visual, auditory or tactile sensory channels.

2. The multi-modal pet interactive panel direct input startup method according to claim 1, characterized in that: Directly enter the set shortcut key through the touch screen, and monitor and identify in real time whether the input shortcut key matches the preset shortcut key to obtain the matching result, including: Send instructions to the touch screen driver to activate the continuous monitoring function of the user's touch input, including the location of the touch point, the type of touch event and the duration of the touch; When the touch screen driver captures a touch event, it receives the corresponding event data and determines whether the touch point is located in the preset shortcut key input area. If the touch event occurs in the shortcut key input area, the touch screen driver reads the character sequence input by the user; Start the regular expression matching process, traverse the preset shortcut key list and match it with the character sequence entered by the user. If the match is successful, execute the operation associated with the matching shortcut key, including starting the pet interaction panel; if there is no matching result after traversing the list, maintain the monitoring state and wait for the user to re-enter to obtain the matching result.

3. The multi-modal pet interactive panel direct input startup method according to claim 2, characterized in that: Based on the matching results, the pet interaction panel is launched and a multimodal interaction interface with multiple interaction modes is displayed, including: Based on the matching results, a start instruction is sent to the pet interactive panel system, including the user identity and interaction mode preference; The pet interactive panel system receives the startup command and extracts the resources of the multimodal interactive interface, including images, audio, video files and interactive logic scripts; After the resource extraction is completed, the multimodal interactive interface is initialized, including setting the interface layout, color, font visual elements and configuring audio output and touch input interaction methods; After initializing the multimodal interactive interface, the pet interactive panel system displays the multimodal interactive interface to the user through the touch screen.

4. The multi-modal pet interactive panel direct input startup method according to claim 3, characterized in that: The multimodal interactive interface allows users to interact with pets in multiple ways, including touch, voice, and vision.

5. The multi-modal pet interactive panel direct input startup method according to claim 4, characterized in that: The touch screen monitors and analyzes the user's interactive operations in real time, including touch and voice input, and performs iterative searches based on the user's interactive operations, simulating the frog's call communication process, constantly listening to and learning the user's preferences, to identify and determine the user's interactive mode, and obtain recognition results, including: Start the real-time monitoring function and voice recognition function of the touch screen. When a touch event or voice input is detected, analyze the user's operation intention, including clicking a button, sliding a slider, and speaking a specific command; Iterative search is performed based on the detected touch events and voice input to simulate the frog's call communication process. During the iteration process, the fitness of the current interaction mode is continuously evaluated to obtain the fitness evaluation result; Based on the fitness evaluation results, learn and identify the user's preferred interaction method. When the preset number of iterations is reached, output the recognition results, including the user's preferred interaction method and commonly used operation instructions.

6. The multi-modal pet interactive panel direct input start method according to claim 5, characterized in that: The calculation formula for the fitness of the current interaction mode is: Among them, F represents the fitness of the current interaction mode; α1, α2, α3, α4 represent weight coefficients; β1, β2, β3, β4, β5 represent adjustment coefficients; T r Represents reaction time; T m represents the maximum acceptable reaction time; δ1, δ2, δ3, δ4, δ5 represent the offset; N i Indicates the number of interactions; N e Indicates the preset number of interactions; T u represents the usage time within the evaluation period; T□ represents the usage time threshold; P p Indicates the proportion of positive feedback; N f represents the total number of feedbacks given during the evaluation period; N n Indicates the minimum feedback quantity.

7. The multi-modal pet interactive panel direct input start method according to claim 6, characterized in that: Based on the recognition results, the content of the multimodal interactive interface is updated in real time to reflect the user's preferred interaction method, and the interaction results are fed back to the user through visual, auditory or tactile sensory channels, including: Generate content that matches the user's preferences based on the recognition results, including changing the arrangement of buttons, displaying different images or animations, and updating the content that matches the user's preferences on the multimodal interactive interface in real time; Displaying content that matches user preferences, including images and animated visual elements, updated in real time through a touch screen to obtain recognition results and interface content; Based on the recognition results and interface content, audio feedback corresponding to the user interaction is generated, including cheers, encouragement, and pet calls.

8. A multi-modal pet interactive panel direct input start system, the system implementing the method as claimed in any one of claims 1 to 7, characterized in that: include: An input recognition module is used to receive and recognize the user's direct input content, including text, symbols, numbers, and user input captured by a touch screen or microphone, including gestures, voice or facial expressions, to obtain a recognition result; A preference analysis module is used to analyze the recognition results and determine the user's preferences and intentions, including the user's preferred interaction method, pet type, and interaction scenario, to obtain analysis results; An interface update module, which is used to update the content of the multimodal interactive interface in real time according to the analysis results of the preference analysis module, including changing the arrangement of buttons, displaying images or animations that match user preferences, and adjusting the color and layout of the interface; A feedback generation module, which is used to generate corresponding audio, visual or tactile feedback according to the interface content, including cheers, encouragement, pet calls, dynamic images or vibration feedback; An output display module, including a touch screen, a speaker, and a vibration motor, is used to display updated interface content to the user and play or provide generated feedback; The control logic module is used to coordinate the work of each module, respond to user input, and provide feedback.

9. A computing device, characterized in that include: one or more processors; A storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the method as claimed in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program, which, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images