A control method for fusion AI pet interaction in pixelated display of mobile phone back screen

Abstract

The application discloses a control method for pixelated display interaction of AI pet on a mobile phone back screen, and relates to mobile terminal interaction technology; the method acquires the state and image data of the AI pet application in real time through a system-level daemon process and inter-process communication; and through a special pixel mapping engine, high-definition data is adaptively converted into a pixel array suitable for display on the back dot matrix screen, and the key emotional characteristics of the pet are mainly retained; voice instructions and sensor postures are fused as multi-modal inputs to trigger pet interaction and realize instant low-delay feedback of the back screen; finally, the pixel data is rendered to the back screen through a bottom-layer driver; the application breaks the dependence of the AI pet on the main screen, realizes all-weather, pixelated and interactive display of the pet on the mobile phone back screen in a locked screen, turned-off screen and the like, enhances the accompanying immersion, and creates a unique pixel art style interactive experience.

Description

Technical Field

[0001] This invention relates to the field of mobile terminal interaction technology, specifically to a virtual avatar interaction control method based on artificial intelligence, and more particularly to a control method that integrates AI pets for pixelated display and interaction on the back screen of a mobile phone. Background Technology

[0002] With the maturity of artificial intelligence and affective computing technologies, virtual companions (AI pets) have become an important functional component of smart terminals such as smartphones. These applications establish emotional connections with users by simulating the behavior, emotions, and learning abilities of living beings, providing companionship and entertainment. Existing AI pets usually run in the form of applications (APPs), and their interaction and display are mainly limited to the device's main screen or specific application interfaces.

[0003] However, this approach has significant limitations. When the device is locked, the app is switched to the background, or the device is flipped (screen down), the AI ​​pet's visual image and real-time status disappear from the user's view, entering a dormant or completely invisible state. This creates a "discontinuity" in the interaction, severely weakening the immersive experience and vitality of the AI ​​pet as an "all-weather companion." Users expect to be able to perceive the pet's presence and status anytime, anywhere, such as glimpsing the pet sleeping on the back of the phone during a work break, or seeing the pet's amusing reactions by shaking the phone. This seamless and continuous companionship experience is something that current technology has failed to meet.

[0004] In recent years, some smartphones have introduced a rear pixel screen, which is usually composed of a low-power LED dot matrix to display simple information such as time and notifications. This provides the hardware possibility for hosting an AI pet as a "second life". However, how to map and interact with an AI pet that runs at the application layer, has complex states and high-definition rendering, in real time, smoothly and with low power consumption on a separate rear pixel screen involves a series of technical challenges such as system-level communication, high-dimensional to low-dimensional visual conversion, multimodal input fusion and underlying drivers. At present, there is still a lack of a systematic solution to realize an all-weather, interactive pixelated life presentation of an AI pet on a rear pixel screen.

[0005] Therefore, there is an urgent need for an innovative control algorithm to connect the entire chain from AI application services to dedicated hardware on the back, and to achieve a seamless interactive experience across screens and states. Summary of the Invention

[0006] This invention aims to solve the problem in the prior art that AI pet interaction is limited to the main screen and cannot provide continuous visual and interactive feedback when the device is locked, running in the background, or flipped. It provides a control method that enables AI pets to be displayed in pixelated form and interact with multimodally in real time on the dot matrix screen on the back of a mobile phone around the clock.

[0007] To achieve the above objectives, the present invention provides a control method for integrating AI pets into the pixelated display interaction of a mobile phone's back screen, comprising the following steps: Step S1, System-level state monitoring and data capture: In the mobile terminal operating system, a system-level daemon process is constructed; the daemon process establishes a connection with the server of the AI ​​pet application through an inter-process communication mechanism, and monitors the internal state change events of the AI ​​pet in real time; and / or, directly captures the image data frames output by the AI ​​pet rendering engine. Step S2, High-dimensional to low-dimensional adaptive pixel mapping: The high-definition image data or motion description data captured in step S1 is input into a pixel mapping engine; the pixel mapping engine performs feature extraction, key frame simplification and color quantization processing on the input data to generate a pixel array adapted to the resolution and color depth of the back dot matrix screen, while retaining the key emotional features of the AI ​​pet. Step S3, Multimodal Interaction Triggering and Closed-Loop Feedback: Listen for multimodal input events from the mobile terminal system layer; when a preset interactive input event is detected, trigger the AI ​​pet to generate a corresponding feedback action, and simultaneously mark the trigger command as high priority, driving the pixel mapping engine to immediately process the data corresponding to the feedback action and generate a real-time pixel array; Step S4, Underlying Driver and Rendering Display: The pixel array generated in step S2 or step S3 is written to the display driver chip of the rear dot matrix screen through the driver interface of the operating system kernel, and the dot matrix light module is controlled to display according to the pixel array, so that the AI ​​pet is continuously displayed on the rear dot matrix screen when the mobile terminal is locked, off, or flipped.

[0008] Furthermore, in step S1, the inter-process communication mechanism includes Binder, AIDL, or shared memory; the internal state change events include changes in emotional state, action execution instructions, and physiological state updates.

[0009] Furthermore, in step S2, the operations performed by the pixelation mapping engine include: locking the core emotional region in the AI ​​pet image based on face or target key point detection technology; using a high-resolution sampling strategy for the core emotional region and a low-resolution sampling or solid color filling strategy for non-core regions; and using a color palette based on the target dot matrix screen color space and an error diffusion algorithm for color quantization.

[0010] Furthermore, in step S3, the multimodal input events include voice command events and sensor data events; the voice command events are obtained by monitoring the recognition results of the system's voice assistant; the sensor data events are obtained by monitoring the data from the accelerometer and gyroscope and determining the attitude changes of the mobile terminal.

[0011] Furthermore, the method also includes an energy efficiency optimization step: defining a low-power sleep state and a corresponding simplified pixel array for the AI ​​pet; when the daemon process does not detect a state change event and does not receive multimodal interactive input within a preset time, controlling the back dot matrix screen to display the simplified pixel array.

[0012] Furthermore, the multimodal interaction trigger also includes haptic feedback: when the dot matrix screen on the back displays a specific interactive animation, the linear motor of the mobile terminal is synchronously triggered to generate vibrations in the corresponding mode.

[0013] Furthermore, the AI ​​pet can be replaced with a virtual assistant image, a digital human image, or a user-defined 3D virtual image.

[0014] Compared with the prior art, the technical solution provided by the present invention has the following beneficial effects: 1. Achieved 24 / 7 emotional companionship: Through system-level guardian process and underlying driver rendering, the AI ​​pet's dependence on the main screen is broken, so that the pet can still "survive" on the back dot matrix screen and display its dynamics when the screen is locked, during calls, when switching applications, or even when the phone is upside down on the desktop, greatly enhancing the continuity and immersion of the emotional connection.

[0015] 2. Created a novel pixel art style interactive experience: The adaptive pixel mapping algorithm is not a simple image compression, but a feature-preserving process that transforms the high-definition AI pet image into a dot matrix pattern with a retro pixel aesthetic. It maximizes the preservation of the pet's expression and emotional characteristics with limited hardware resources, forming a unique visual style.

[0016] 3. A low-latency multimodal interactive closed loop has been constructed: voice commands and motion-sensing interaction are deeply integrated as inputs and drive the back screen to achieve millisecond-level visual feedback, forming a smooth "input-processing-output" closed loop; the user's natural interaction with the mobile phone (such as speaking, picking up, shaking) can be directly converted into control of the pet on the back screen, making the interaction more intuitive and vivid.

[0017] 4. Balancing functionality and system energy efficiency: Through intelligent state management (such as low-power state switching) and efficient data processing pipelines (such as shared memory and circular buffers), the solution provides a continuous interactive experience while minimizing the consumption of system power consumption and computing resources, ensuring the practicality and deployability of the solution. Attached Figure Description

[0018] Figure 1 This is a block diagram of the overall system architecture of the control method provided in the embodiments of the present invention; Figure 2 This is a schematic diagram of the processing flow of the pixelation mapping engine provided in an embodiment of the present invention; Figure 3 This is a flowchart of the multimodal interaction closed loop provided in the embodiments of the present invention; Figure 4 This is a flowchart of the complete control method provided in the embodiments of the present invention. Detailed Implementation

[0019] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the following description is intended to provide an in-depth explanation of the principles of this invention, rather than to limit the invention. Other equivalent or modified embodiments can be conceived by those skilled in the art after reading this specification, and these should all be included within the protection scope of this invention.

[0021] This invention provides an AI pet interaction system and its control method for a dot matrix screen on the back of a mobile terminal. The core of this invention lies in creatively solving the "performance gap" problem between high-definition, complex AI pet applications and low-resolution, monochrome or low-color dot matrix screens by using a middleware adaptation layer (i.e., a system daemon process) located between the operating system application layer and the hardware driver layer. Furthermore, this technology integrates voice and sensor input to build an efficient, energy-saving, and immersive multimodal interactive closed loop.

[0022] The following will combine Figures 1 to 4 Taking a specific implementation scenario as an example, this embodiment elaborates on the technical solution of the present invention in depth. The hardware platform is a smartphone with a rear LED dot matrix screen (e.g., 16x16 pixels, supporting 4 levels of grayscale), and the interactive object is an AI pet application called "Virtual Pet".

[0023] First refer to Figure 1 The system demonstrates the overall architecture of the present invention, which is mainly divided into four layers: AI pet service layer 101, middleware adaptation layer 102, system interaction layer 103 and hardware driver layer 104. The system is not a single application, but a layered and decoupled collaborative working system.

[0024] 1. The AI ​​Pet Service Layer 101 is the source of pet status and data. In this embodiment, the "Virtual Pet" APP needs to open its internal state according to the agreed interface. The APP implements an AIDL interface called IPetService, which defines a state enumeration (PetState), such as IDLE, HUNGRY, HAPPY, SLEEPING, PLAYING, etc., and provides a callback interface onPetFrameUpdate(Bitmap frame, int state). When the pet's internal state machine changes due to time, user interaction, or internal logic (for example, from IDLE to HUNGRY), or when the rendering engine completes the drawing of a frame of animation, the APP will actively send out the status code and / or the Bitmap image data of the current frame through the Binder IPC mechanism. In order to pursue the ultimate performance, the Bitmap data can also be transferred by sharing memory (MemoryFile) to avoid the overhead of cross-process data copying.

[0025] The AI ​​pet service layer 101 is the source of interactive content, such as a standalone "virtual pet" application. This application implements the complete logic of the pet, including a state machine (such as mood and hunger value), a behavior tree (which determines the pet's response to stimuli), and a high frame rate, high resolution animation rendering engine. Its core responsibility is to produce high-definition bitmap sequences that express the pet's emotions and actions. In this implementation, the application communicates with the lower layer through a predefined local interface (such as Android's AIDL interface or Unix Domain Socket) rather than directly manipulating the hardware. This design allows any AI pet application that conforms to the interface specification to access this system, achieving separation of application and hardware adaptation, and greatly improving the system's compatibility and scalability.

[0026] 2. The middleware adaptation layer 102 is the core innovation layer of this invention. It resides in memory as a daemon process with system privileges, acting as a "translator" and "scheduling center." This daemon process contains the following key modules: 1) Data Reception and Buffering Module 201: Listens for Binder callbacks from IPetService; received Bitmap data is placed into a double buffer or circular buffer to balance the speed difference between production (APP rendering) and consumption (pixelation processing) to avoid frame loss; this module is responsible for asynchronous communication with upper-layer applications and system services; it maintains one or more shared memory buffers or high-priority message queues; when a new bitmap data frame is received from the AI ​​Pet application, or an action command is received from the system voice service, this module will place it into the corresponding buffer and add a timestamp and priority label; for sensor events, it registers listeners through system APIs (such as Android's SensorManager), and after receiving raw sensor data streams that meet preset conditions (such as acceleration thresholds, specific attitude angles), it performs preliminary filtering and feature extraction, transforming it into an abstract "interactive event" (such as "SHAKE" or "FACE_UP").

[0027] 2) Pixelation Engine 202: This is the key to converting high-definition pet images into dot matrix screen data. Its internal operation process (corresponding to...) Figure 2 It is extremely meticulous, not a simple scaling up, but a re-creation process that preserves the essence of the work, combining... Figure 2 As shown, Figure 2 This study reveals the key transformation steps from a high-resolution bitmap to an expressive low-resolution pixel image. This dynamic and adaptive process aims to maximize the transmission of the original image's emotional core on a highly information-constrained medium. A dedicated "pixelation mapping engine" is designed to convert captured high-resolution pet images (or vector motion data) into pixel arrays adapted to the resolution and color depth of pixel-based screens in real time through algorithms such as feature extraction, keyframe simplification, and color quantization. Key emotional features of the pet, such as its eyes and mouth shape, are preserved. Specifically, this includes: a) ROI (Region of Interest) Locating (Step S201): The engine first performs real-time analysis on the input high-resolution pet bitmap; the engine uses lightweight computer vision algorithms (such as region detection based on Haar features or lightweight neural networks) to quickly locate key emotional regions in the image, the most important of which are the eyes and mouth; in cartoonish and stylized AI pet design, the size, shape, and highlight position of the eyes, as well as the curvature of the mouth, are the most critical features for expressing emotions such as happiness, surprise, and drowsiness; the engine generates one or more "regions of interest" masks for these areas; the significance of this step is to establish the priority of subsequent processing, ensuring that limited pixel resources are used first to depict the most expressive parts.

[0028] For example, input a 640x640 resolution color bitmap of a pet; first, use a lightweight keypoint detection model (such as the MobileNet-SSD adapted version) or traditional image processing algorithms to locate the core feature regions of the pet's face, such as the left eye region R_eye, the right eye region R_eye, and the mouth region R_mouth; these regions are key to expressing emotions.

[0029] b) Adaptive Region Sampling (Step S202): This step abandons the traditional approach of uniform downsampling the entire image. Based on the ROI locked in step S201, the engine divides the image into a virtual, non-uniform grid: For high-priority ROIs such as eyes and mouths, a high-density sampling grid is used; for example, in the area where the eyes are located, a 4x4 sub-grid may be used to sample the original image, so that multiple pixels can be used to depict the contour, pupil and highlights of the eyes, and even the movement of "catchlight" can be simulated by the brightness and darkness changes of some pixels; For secondary areas such as the body and background, a low-density sampling grid is used; for example, the entire body may only be sampled using a large grid unit (such as 2x2) to obtain a general contour and brightness relationship, or only a single point to represent its centroid; This "non-uniform adaptive sampling" strategy is the optimal solution under strict pixel budget constraints. It is essentially an intelligent allocation of visual information, which can play a role similar to "using good steel on the blade".

[0030] For example, the target output is set to a 16x16 pixel array; for the core emotional regions R_eye and R_mouth, a higher density sampling is used; for example, the R_eye region (approximately 30x30 pixels in the original image) is mapped to a 2x2 pixel region of the target array, and a local pixel value weighted average is used to ensure that highlights (catcheyes) are preserved; for non-core regions (such as the body and background), a more aggressive downsampling is used, such as mapping a large body region to a single pixel, or directly filling it with a single background color (such as black) to highlight the subject.

[0031] c) Color Quantization and Dithering (Step S203): Since the target dot matrix screen is usually monochrome (black and white) or has only a few colors (such as 16 levels of grayscale), the sampled RGB color values ​​need to be significantly quantized; simple direct quantization will produce severe color banding and loss of detail; therefore, this invention applies an improved error diffusion dithering algorithm (such as the Floyd-Steinberg algorithm), which, when processing the current pixel, diffuses the quantization error between its color and the nearest available color (such as black or white) to adjacent, unprocessed pixels in a certain proportion; Figure 2In the example, a pixel at the chin that should have been dark gray was quantized to white (turned off) due to resource constraints. Its error (a negative value) was spread to the pixels to the lower right and below, making these pixels more likely to be quantized to black (turned on). This maintained the overall grayscale feel of the area on a macroscopic level, simulating the smooth gradients and texture details in the original image. This is a key step in achieving a "lifelike" feel.

[0032] For example, the target dot matrix screen may only support 4 levels of grayscale (black, dark gray, light gray, and white); the engine has a built-in optimized 4-color palette and uses the Floyd-Steinberg error diffusion algorithm to map the RGB value of each "super pixel" obtained in step S202 to the closest color in the palette and diffuse the quantization error to neighboring, unprocessed pixels; in this way, at low color depth, the visual mixing effect of the human eye can be used to simulate a smoother grayscale transition and avoid obvious color stratification (contour).

[0033] d) Data formatting (step S204): The engine packages the processed two-dimensional pixel matrix (each pixel is 1 or 4 bits of data) according to the format required by the hardware driver; for example, for a 16x16 monochrome screen, a 32-byte array (16... (16 / 8); For grayscale screens, data needs to be organized according to driver requirements; the formatted data is sent to the output buffer, waiting to be sent.

[0034] For example, the final 16x16 two-dimensional pixel array is generated, with each pixel represented by 2 bits (corresponding to 4 gray levels), and packed into a byte array of length 64 bytes (byte

[64] ) in row-major order. This array is the original data that can directly drive the dot matrix screen.

[0035] 3. System interaction layer 103 is responsible for processing input from other parts of the system and forming an interactive closed loop; it listens for wake words and commands from the system's voice assistants (such as Xiao Ai, Siri, etc.). When the user issues an interactive command (such as "Dance!" or "Are you happy?"), the AI ​​pet generates a feedback action, and the algorithm immediately pushes the pixel sequence corresponding to that action to the back screen; combined with accelerometer and gyroscope data, when the phone is detected to be picked up, flipped, or shaken, the pet's corresponding reaction is triggered (such as looking around or a dizziness effect), and is simultaneously displayed on the back screen; combined with... Figure 3 As shown, this layer mainly includes two channels: 1) Voice Command Channel: The daemon process registers a listener with the system voice service (such as Android's VoiceInteractionService); the daemon process captures specific voice commands by listening to system broadcasts or collaborating with intelligent voice assistants (such as the phone's built-in voice assistant); when the user says to the phone, "Xiao Ai, make the pet dance," the system voice service recognizes the intent (PET_INTERACT) and action parameters (ACTION_DANCE); this command is passed to the daemon process, which immediately calls the performAction("DANCE") method of IPetService through Binder; for example, when the user says When the voice assistant says, "Hey, [assistant name], play with my pet," after recognizing the main command, it can pass a preset command code (such as ACTION_PLAY) to the daemon process through a lightweight callback interface. This process is imperceptible to the user, achieving seamless integration of voice assistant functionality and pet interaction. At the same time, the daemon process internally marks a "high-priority refresh" flag. After receiving the command, the AI ​​pet app starts executing the "dance" animation sequence. At this time, once the data receiving module 201 of the middleware adaptation layer 102 receives the dance frame data from the app, the pixelation mapping engine 202 will interrupt the current normal processing flow and prioritize the processing of these high-priority frames to ensure the real-time nature of the interactive response.

[0036] 2) Sensor Linkage Channel: This mainly refers to the device's inertial measurement unit, including the accelerometer and gyroscope. The system constructs a system-level daemon process, which establishes a long connection with the AI ​​pet application service through IPC mechanisms such as Binder / AIDL. The daemon process continuously acquires low-power sensor data through the system service. For example, by analyzing accelerometer data, it can detect specific gestures such as "picking up," "tap twice," or "shaking" the phone. By combining gyroscope data, it can determine whether the phone is in a "screen-up" or "inverted" position. These physical interactions are defined as "metaphorical" commands that trigger specific pet reactions. The system monitors pet state change events in real time (such as mood changes, action execution, and physiological state updates) or directly captures the current frame Bitmap data output by the pet rendering engine. The daemon process registers a SensorEventListener to monitor accelerometer and gyroscope data. Through the following simple threshold judgment and state machine, it achieves the mapping from posture to pet action: a) When the device is detected to be picked up quickly from a horizontal stationary state (Z-axis acceleration change exceeds the threshold), it is mapped to the pet's "peeking out" action; b) When the device is detected to be continuously oscillating slightly around the Y-axis (simulating being petted), it is mapped to the pet's "comfortably squinting" action; c) When the device is detected to be violently shaken (highest value of the three-axis acceleration vector), it is mapped to the pet's "dizziness and seeing stars" effect; After the sensor data is mapped, a virtual "action command" is generated. This command is also sent to the AI ​​pet service layer 101 to trigger the corresponding animation and follow the "high-priority refresh" path, ultimately driving the back screen display.

[0037] Figure 3 It demonstrates how voice and sensor input can be organically integrated to form a low-latency, high-priority interactive response loop, which constitutes the source of "spirit" in the user experience.

[0038] When a user interacts with the device via voice (such as saying "Let's dance!") or physical action (such as picking up the phone to watch), the event is captured by the system interaction layer and immediately passed to the data receiving and buffering module of the middleware adaptation layer. This module marks such interactive events as "high priority" and may include specific action parameters (such as ACTION_DANCE). Subsequently, the module will not wait for the normal state update frame that may be being processed, but will immediately interrupt or pause the normal processing pipeline and send the action parameters and priority flags to the AI ​​pet application. After receiving the high-priority instruction, the AI ​​pet application will interrupt the current idle or loop animation, immediately call the corresponding "dance" animation resource, and start generating a continuous high-definition animation frame sequence that expresses the dance action, and send it to the daemon process at the highest possible frame rate. When the pixelation mapping engine receives these frames marked with "high priority", it will activate a fast channel: it may simplify the region detection in S201 (using the previous result or a preset template) and prioritize the use of computing resources for processing, ensuring that the first frame of dancing motion is rendered on the back dot matrix screen within a very short latency (e.g., <100ms). This closed loop of "perception-decision-high priority response-display" makes the pet seem to understand the user's intentions in real time and give feedback, greatly enhancing the realism of the interaction and the user's sense of participation.

[0039] 4. The hardware driver layer 104 is responsible for the final execution; this layer is the final instruction execution layer and is usually provided by the device manufacturer. The daemon process writes the formatted pixel byte array to the character device node (e.g., / dev / pixel_screen) provided by the Linux kernel (write()). The dedicated driver chip of the dot matrix screen (e.g., TM1640) reads the data from this node and controls the corresponding LED dot matrix to turn on and off, thus presenting a dynamic pixel pet image on the back of the phone. The processed pixel array is written to the kernel node, driving the dot matrix light module to display in real time, ensuring that the pet still "lives" and interacts with the user when the screen is locked, the screen is black, or even when the phone is upside down.

[0040] This invention interacts with the driver of the back dot matrix screen through a standard Linux driver interface (such as / sys / class / leds / back_matrix or a device file that communicates via the I2C bus); the protection process controls the on / off state of each LED by writing the well-formatted pixel array (usually a two-dimensional array or a one-dimensional byte stream) generated by the pixel mapping engine to a specific driver node, thereby presenting the final image on the screen.

[0041] like Figure 4 As shown in the diagram, this illustrates the overall process of implementing the system from startup to continuous operation, depicting the complete control flow from system startup to entering a stable working state and continuously processing various events: S401, System startup, loading daemon process: After the mobile terminal operating system completes the boot process, the core of this invention—the system daemon process—is automatically started as a system service. This process can be configured to be called in init.rc or systemd service to ensure that it is ready before the user unlocks the screen. After the process starts, it first loads the configuration file and initializes the global data structure, such as allocating independent circular buffers for frame data of different priorities and initializing the hardware file descriptor for communication with the dot matrix screen driver.

[0042] S402, Daemon process initialization, service connection, and listener registration: This step establishes all communication links, specifically including: 1) Connecting to the AI ​​Pet Application Service: The daemon attempts to connect to the AI ​​Pet application through a predefined IPC mechanism (such as binding to a Service); if the application is not running, it can send a gentle start request; after the connection is established, the two parties negotiate the communication protocol version and establish a shared memory area for transmitting bitmap data and a message queue for transmitting control commands. 2) Register system broadcast listener: The daemon registers with the system broadcast manager to listen for specific broadcast intents related to the voice assistant (e.g., com.example.ACTION_PET_COMMAND). 3) Register sensor listener: The daemon requests access to the accelerometer and gyroscope from the sensor service and registers a SensorEventListener with an appropriate sampling rate (e.g., SENSOR_DELAY_GAME for fast response, or SENSOR_DELAY_UI for power saving); at the same time, it loads a preset interactive posture mode library (such as a set of acceleration threshold modes for recognizing "shaking").

[0043] S403, enter the main loop and listen for multiple events: After initialization, the daemon process enters a non-blocking, multiplexed event loop; the core of this loop is to use mechanisms such as epoll (Linux) or Looper (Android) to listen to multiple event sources at the same time: data sockets from the AI ​​pet application, message queues from system broadcasts, and sensor event callbacks; the loop is designed to be efficient and low-power, and the process can be suspended by the scheduler when there are no events.

[0044] S404, determine if an event has occurred: if it is a "pet status / frame update event" (S405), then execute the regular pixelation mapping process (S406); if it is a "voice / sensor interaction event" (S407), then trigger the AI ​​pet action and enter the high-priority pixelation mapping process (S408); if it is "entering a low-power condition" (such as no interaction for a long time) (S409), then generate and output a static / simple sleep frame (S410); this is the decision point in the loop; when the listener is triggered, the process will determine the event type and route it to different processing branches; events are usually divided into three categories with clear processing priorities: high-priority interaction events > regular status update events > low-power management events.

[0045] An extension to S405 (pet status / frame update event): This is the most common path, handling animation frames generated by the pet's autonomous behaviors (such as breathing, blinking, wandering). When a new bitmap data frame is received, the daemon places it into the regular frame buffer. The engine processes frames from the buffer according to a preset rhythm matched to the pixel screen's refresh rate (e.g., 10fps). If the buffer becomes too cluttered (indicating the pet animation is too fast or the engine is processing too slowly), a frame dropping strategy is initiated to ensure timely display. In this path, the pixel mapping engine performs a complete and detailed process. Figure 2 process.

[0046] Extensions for S407 (Voice / Sensor Interaction Events): This is a high-response path; when such an event arrives, the daemon immediately sets a "high priority" flag in the event loop and may send an interrupt signal to the engine; upon receiving this signal, the engine saves the processing context of the current regular frame and immediately processes the interaction event; specifically: 1) Parse the event parameters, generate a high-priority control message and send it to the AI ​​pet application to request the execution of a specific action; 2) When the application returns the first frame image of the action, the engine opens a "fast track" for it, which may adopt a simplified but fast mapping strategy (e.g., using a simplified sampling template pre-computed for the action, or reducing the number of iterations of the dithering algorithm) to sacrifice a little image quality in exchange for the ultimate response speed and ensure the immediacy of interactive feedback.

[0047] Extensions for S409 (whether to enter low-power conditions): This is a critical path for energy efficiency management; the daemon contains a dynamic power manager that continuously monitors the following metrics: a) User interaction idle timer: Records the time since the last voice / sensor interaction; b) System status: Is the phone screen off? Is it connected to a charger? What is the battery level? c) Application status: Is the AI ​​pet application running in the background, or has it notified the engine to enter sleep mode? When the conditions are met (e.g., the phone screen is off and there is no physical interaction for 5 minutes), it is determined that it has entered a low-power condition. At this time, the regular animation frame generation and processing will be paused or the frequency will be greatly reduced. Instead, a low-power sleep frame (S410) will be generated / output. It can be a minimalist, static pet outline (represented by only a few pixels) or a slowly flashing "Zzz" symbol animation, with a refresh rate reduced to 1fps or even lower. This sleep frame will be pre-calculated and cached, and will only need to be output in a loop afterward to minimize the power consumption of the dot matrix screen driving circuit. The system will quickly return to full functionality when a new interactive event (such as picking up the phone) "wakes it up".

[0048] S411, write the mapped pixel array to the driver node: regardless of the path used to generate it, the final pixel array will be sent to a display queue; a separate, high-priority I / O thread is responsible for writing the data in this queue to the device file corresponding to the rear dot matrix screen (such as / dev / i2c-3) through the write() system call, according to the timing and format required by the hardware; the writing process needs to take into account the hardware refresh cycle to avoid conflicts.

[0049] S412, driving the dot matrix display on the back of the phone: After receiving the data, the hardware driver latches it into the display cache and drives the LED dot matrix to light up row by row or column by column according to its own scanning logic, eventually forming a stable image in the human eye; at this point, a complete processing cycle ends; the system then immediately returns to step S403 to wait for the next event, thus forming a continuous, intelligent, adaptive and energy-saving interactive loop.

[0050] ( Specific Implementation Assuming the user has installed a "virtual electronic cat" application and enabled the system of this invention; when the phone is placed face down on a table: 1. Under normal conditions: The pet sleeps on the screen; the application generates breathing animation frames of the sleeping cat at 1fps; the engine processes these frames and simulates the slight undulation of the cat's body with periodic brightness changes of a few pixels on the 16x16 dot matrix screen on the back.

[0051] 2. During user interaction: When a user picks up their phone, the sensor detects the "picked up" gesture; a high-priority event is triggered; the system immediately notifies the pet app, and the pet "wakes up"; the engine quickly renders the first frame of the awakened pet (sleepy-eyed) onto the back screen; the user then says to the phone, "Let's dance!", and the voice command is captured; the pet app immediately plays a dance animation (8fps); the engine prioritizes processing these animation frames, presenting a simple yet rhythmic pixel dance on the back screen; the entire process has extremely low latency, and the pet reacts swiftly.

[0052] 3. Low power mode: When the user puts the phone back in their pocket, after one minute of no interaction, the system determines that it has entered low power mode. The pet application stops sending animation frames, and the engine switches to outputting a static cat paw print icon made up of only 4 pixels, which flashes once per minute. The power consumption of the dot matrix screen on the back drops to a negligible level until the user takes the phone out again, and the pet instantly "comes back to life".

[0053] ( Extended Example 1 Haptic feedback linkage: While driving the display in step S411, the following steps can be added: Based on the currently displayed pixel animation content (such as the pet "jumping"), the vibration mode (VibrationEffect) with a specific frequency and duration is called through the Android Vibrator service to achieve an enhanced experience of "visual-tactile" linkage.

[0054] ( Extended Example 2 Cross-device social interaction: When two mobile phones equipped with this solution discover each other via NFC or Bluetooth LE and are placed back to back, their guardian processes can exchange pet data through P2P communication; briefly "project" the pixelated image of one's own pet onto the back screen of the other's mobile phone, stay for a few seconds, and then return, realizing the social gameplay of "pet visiting each other's homes".

[0055] ( Extended Example 3 AR virtual-real fusion: When the user turns on the main camera and AR function, the main screen displays the AR pet moving in the real scene; at the same time, the guardian process can extract the simplified outline or footprint information of the AR pet and map it to the back screen display, forming an immersive experience that links the content of the front and back screens.

[0056] Through the above systematic design and in-depth optimization process, this invention successfully realizes an AI pet experience that is emotionally rich, responsive, interactive, and power-controlled on the dot matrix screen on the back of a resource-constrained mobile device, greatly expanding the imagination and application value of side interaction on mobile terminals.

[0057] It should be understood that the above description is only a preferred embodiment of the present invention and is not sufficient to limit the technical solution of the present invention. For those skilled in the art, within the spirit and principles of the present invention, additions, subtractions, substitutions, transformations or improvements can be made based on the above description, and all such additions, subtractions, substitutions or improvements should fall within the protection scope of the appended claims of the present invention.

Claims

1. A control method for integrating AI pets into the pixelated display and interaction of a mobile phone's back screen, characterized in that, The method, applied to a mobile terminal equipped with a rear dot-matrix screen, includes: System-level status monitoring and data capture steps: Through a daemon process running at the mobile terminal operating system layer, the AI ​​pet application is interacted with using inter-process communication mechanisms to obtain the status data or image frame data of the AI ​​pet in real time. Adaptive pixelation mapping step: The acquired state data or image frame data is input into the pixelation mapping engine for processing to generate pixelation display data that is adapted to the resolution and color depth of the rear dot matrix screen; Multimodal interaction triggering steps: Listen for at least one preset type of interactive input event from the mobile terminal system layer; Real-time feedback-driven steps: In response to the detected interactive input event, the AI ​​pet is triggered to generate corresponding feedback data, and corresponding pixelated display data is generated based on the feedback data; The underlying rendering and display steps are as follows: The pixelated display data is written into the driver interface of the rear dot matrix screen to drive the rear dot matrix screen to display the corresponding pixelated image, so that the AI ​​pet is continuously displayed on the rear dot matrix screen when the main screen of the mobile terminal is inactive.

2. The control method for integrating AI pets for pixelated display and interaction on the back screen of a mobile phone according to claim 1, characterized in that, In the system-level state monitoring and data capture steps, the inter-process communication mechanism includes the Binder interface, AIDL interface, or shared memory; the acquired data includes the internal state change events of the AI ​​pet and / or bitmap data output by the AI ​​pet's application rendering engine.

3. The control method for integrating AI pets for pixelated display and interaction on a mobile phone back screen according to claim 1, characterized in that, The adaptive pixelation mapping step specifically includes: The key emotional region detection is performed on the input image frame data to identify at least one core emotional region in the AI ​​pet image; The core emotional region is processed using a first sampling strategy, and the non-core emotional region is processed using a second sampling strategy. The sampling density of the first sampling strategy is higher than that of the second sampling strategy. Using a limited color palette configured for the rear dot matrix screen, color quantization and error diffusion processing are performed on the sampled image data to obtain the pixelated display data.

4. The control method for integrating AI pets for pixelated display and interaction on a mobile phone back screen according to claim 3, characterized in that, The key emotional regions include the eye region and the mouth region; the color quantization and error diffusion processing adopts the Floyd-Steinberg error diffusion algorithm.

5. The control method for integrating AI pets for pixelated display and interaction on a mobile phone back screen according to claim 1, characterized in that, In the multimodal interaction triggering step, the preset type of interactive input event includes voice command events and sensor posture change events; wherein, The monitoring of voice command events includes: monitoring the recognition results of the system's voice assistant, and determining that a voice command event has been received when the recognition result matches the preset pet interaction command; Monitoring sensor attitude change events includes: monitoring data from the accelerometer and gyroscope, determining the attitude action of the mobile terminal based on data changes, and determining that a sensor attitude change event has been received when the attitude action matches a preset interactive attitude.

6. The control method for integrating AI pets for pixelated display and interaction on a mobile phone back screen according to claim 5, characterized in that, In the real-time feedback driving step, in response to the interactive input event, the method further includes: The generated feedback data is marked as high priority. The pixelation mapping engine prioritizes feedback data marked as high priority, interrupting or delaying the processing of regular status data or image frame data.

7. The control method for integrating AI pets for pixelated display and interaction on a mobile phone back screen according to claim 1, characterized in that, The method also includes energy efficiency management steps: If no status update of the AI ​​pet is obtained and no interactive input event is detected within a first preset time period, the back dot matrix screen is controlled to display a preset low-power static image or dynamic image. The amount of pixelated display data corresponding to the low-power static image or dynamic image is less than that of the pixelated display data in the normal interactive state.

8. The control method for integrating AI pets for interactive display on the pixelated back screen of a mobile phone according to any one of claims 1 to 7, characterized in that, The real-time feedback driving step also includes a haptic feedback step: Based on the interactive content corresponding to the feedback data or the final generated pixelated display data, generate corresponding haptic feedback instructions; The linear motor of the mobile terminal is controlled to execute the haptic feedback command to generate a vibration effect synchronized with the content displayed on the dot matrix screen on the back.

9. The control method for integrating AI pets for interactive display on the pixelated back screen of a mobile phone according to any one of claims 1 to 7, characterized in that, The AI ​​pet can be a virtual assistant, a digital human, or a user-defined 3D virtual avatar.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the control method for interacting with a fusion AI pet displayed in a pixelated manner on the back screen of a mobile phone as described in any one of claims 1 to 9.

Images