Synchronization method and system for vehicle terminal

Abstract

The application discloses a kind of synchronization method and system of vehicle terminal, belong to action synchronization technical field.The application obtains imported virtual initial model, obtains action sequence data and expression animation data according to the virtual initial model;Response operation instruction of the interactive terminal, adjusts the action sequence data and / or expression animation data according to the operation instruction, obtains target data information;The target data information is sent to the vehicle display terminal by the communication module, to make the vehicle display terminal according to the target data information synchronous demonstration action and / or expression.It realizes the real-time synchronous comparison of the action of vehicle display terminal.

Description

Technical Field

[0001] This invention relates to the field of motion synchronization, and more particularly to a synchronization method and system for an in-vehicle terminal. Background Technology

[0002] Currently, the application scenarios of vehicle-mounted robots are becoming increasingly widespread, and their expressions and movements are becoming more and more varied. The process of designing the head movements of a vehicle-mounted robot generally involves first calculating a complete sequence of movements on a PC, then transmitting the sequence of movements to the robot's storage chip. The robot then loads the movement parameters and controls its head to rotate, recreating the sequence of movements, which the designer then observes and verifies.

[0003] Currently, the design process for head movements of vehicle-mounted robots is quite cumbersome. When designers design movements on a PC, they cannot simultaneously compare the movement effects of the actual vehicle-mounted robot in real time. Summary of the Invention

[0004] The main objective of this invention is to provide a synchronization system and method for vehicle-mounted terminals, aiming to solve the problem of how to achieve synchronous comparison of the actions of vehicle-mounted display terminals.

[0005] To achieve the above objectives, the present invention provides a synchronization method for an in-vehicle terminal. This method is applied to an in-vehicle terminal synchronization system, which includes an interactive terminal, a communication module, and an in-vehicle display terminal. The interactive terminal runs an action design program, and the communication module enables information interaction between the interactive terminal and the in-vehicle display terminal. The synchronization method includes: Obtain the imported virtual initial model, and obtain action sequence data and facial animation data based on the virtual initial model; In response to the operation command of the interactive terminal, the action sequence data and / or facial animation data are adjusted according to the operation command to obtain target data information; The target data information is sent to the vehicle-mounted display terminal through the communication module, so that the vehicle-mounted display terminal can synchronously display actions and / or expressions based on the target data information.

[0006] Optionally, responding to the operation command of the interactive terminal and adjusting the action sequence data and / or facial animation data according to the operation command to obtain target data information includes: When the operation instruction is a drag instruction, the drag instruction is parsed to obtain the operation information of the virtual initial model; Adjust the action sequence data and / or facial animation data according to the operation information to obtain target data information.

[0007] Optionally, adjusting the action sequence data according to the operation instructions to obtain target data information includes: The action sequence data is adjusted according to the operation instructions to obtain the adjusted action sequence data; Based on the correspondence between action sequence data and facial expression animation data, determine the first facial expression animation data corresponding to the adjusted action sequence data; Based on the adjusted action sequence data and the first facial expression animation data, target data information is obtained.

[0008] Optionally, before determining the first facial animation data corresponding to the adjusted action sequence data, the following steps are included: The adjusted action sequence data is matched with the standard action sequence data; If the adjusted motion sequence data matches the standard motion sequence data, then the step of executing the first facial animation data corresponding to the adjusted motion sequence data is performed; If the adjusted action sequence data does not match the standard action sequence data, an error message will be displayed through the interactive terminal.

[0009] Optionally, after sending the target data information to the vehicle-mounted display terminal via the communication module so that the vehicle-mounted display terminal synchronously displays actions and / or expressions based on the target data information, the method further includes: The actions and / or expressions are rendered, and the interaction stops when the rendered actions and / or expressions are displayed synchronously.

[0010] Optionally, the data information transmission adopts a data stream caching strategy, including: The system receives the data information, converts it into a data stream, implements a corresponding caching strategy, and then transmits it.

[0011] Furthermore, to achieve the above objectives, the present invention provides a synchronization system for an in-vehicle terminal. The synchronization system includes an interactive terminal, a communication module, and an in-vehicle display terminal. The interactive terminal runs an action design program, and the communication module enables information interaction between the interactive terminal and the in-vehicle display terminal, including: The system includes an interactive terminal, a communication module, and a vehicle-mounted display terminal. The interactive terminal runs an action design program and uses the communication module to enable information interaction between the interactive terminal and the vehicle-mounted display terminal. The interactive terminal is used to acquire the imported virtual initial model and obtain action sequence data and facial animation data based on the virtual initial model. The interactive terminal is used to respond to the operation instructions of the interactive terminal, and adjust the action sequence data and / or facial animation data according to the operation instructions to obtain target data information; The interactive terminal is used to send the target data information to the vehicle-mounted display terminal through the communication module, so that the vehicle-mounted display terminal can synchronously display actions and / or expressions according to the target data information.

[0012] Optionally, the interactive terminal is also used to store head motion sequences and facial expression animations, call a motion design program, call the head motion sequences and facial expression animations through the motion design program, and send the head motion sequences and facial expression animations to the vehicle-mounted display terminal for playback.

[0013] Optionally, the interactive terminal is connected to the communication module via a serial port, and the communication module is connected to the vehicle-mounted display terminal via a CAN bus; The communication module is used to transmit the target data information to the vehicle-mounted display terminal.

[0014] Optionally, the communication module is further configured to convert the target data information into a data format supported by the vehicle-mounted display terminal, and send the converted target data information to the vehicle-mounted display terminal.

[0015] This invention acquires an imported virtual initial model, and obtains action sequence data and facial animation data based on the virtual initial model; responds to the operation instructions of the interactive terminal, adjusts the action sequence data and / or facial animation data according to the operation instructions to obtain target data information; and sends the target data information to the in-vehicle display terminal through the communication module, so that the in-vehicle display terminal synchronously displays actions and / or expressions according to the target data information. Thus, the virtual initial model is acted upon according to the operation instructions, while data information is transmitted to the in-vehicle display terminal, achieving real-time synchronous comparison of the actions of the in-vehicle display terminal. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating the first embodiment of the synchronization method for the vehicle-mounted terminal of the present invention. Figure 2 This is a schematic diagram of the structure of the first embodiment of the synchronization method for the vehicle-mounted terminal of the present invention; Figure 3 This is a flowchart illustrating the second embodiment of the synchronization method for the vehicle-mounted terminal of the present invention. Figure 4 This is a flowchart illustrating the third embodiment of the synchronization method for the vehicle-mounted terminal of the present invention. Figure 5 This is a flowchart illustrating the fourth embodiment of the synchronization method for the vehicle-mounted terminal of the present invention. Figure 6 This is a flowchart illustrating the fifth embodiment of the synchronization method for the vehicle-mounted terminal of the present invention. Figure 7 This is a schematic diagram of the data layer definition framework for the synchronization method of the vehicle terminal of the present invention; Figure 8 This is a flowchart illustrating the sixth embodiment of the synchronization method for the vehicle-mounted terminal of the present invention. Figure 9 This is a schematic diagram defining the operation framework of the synchronization method for the vehicle-mounted terminal of the present invention; Figure 10 This is a schematic diagram of the first embodiment of the synchronization system of the vehicle-mounted terminal of the present invention.

[0017] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0018] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0019] This invention provides a synchronization method for an in-vehicle terminal, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of a synchronization method for an in-vehicle terminal according to the present invention.

[0020] It should be noted that the application scenario of this embodiment is that designers use the synchronization method of the vehicle terminal to synchronize the actions of the physical vehicle robot and the virtual initial model.

[0021] In this embodiment, the synchronization method of the vehicle terminal includes: Step S10: Obtain the imported virtual initial model, and obtain action sequence data and facial animation data based on the virtual initial model.

[0022] It should be noted that the execution subject of this embodiment can be the synchronization device of the vehicle terminal, or other devices that can achieve the same or similar functions. This embodiment does not limit this; this embodiment takes the synchronization device of the vehicle terminal as an example for explanation.

[0023] The vehicle-mounted terminal's synchronization equipment consists of hardware and software. The hardware includes a computer host, 3D sensors, a camera, and suitable input devices (such as a keyboard and mouse), as well as a Simbox (a communication box, an embedded device that can communicate with a PC serial port device and the vehicle's CAN network interface device, supporting format conversion between PC serial port message data and vehicle CAN message data) for data processing and transmission. The software includes 3D design tools (such as Unity3D and Unreal Engine) running on the computer host, design tool plugins, and algorithms for parsing operation commands.

[0024] It should be understood that the virtual initial model is a highly intelligent 3D model that includes not only the external shape of the vehicle robot, but also rich motion and facial expression information. Motion data includes flexible head rotation and eye gaze movements, while facial expression animation data includes smiles, grins, surprise, etc.

[0025] Action sequence data is a form of data that records a series of continuous actions or events. These actions or events can be found in various fields, such as motion analysis, behavior recognition, and industrial automation. In action sequence data, each action or event typically consists of a timestamp, action type, and possible additional information. Facial expression animation data is a form of data that records changes in human facial expressions. It contains various facial expression states and changes over a period of time, usually obtained by collecting facial movements or simulating them using 3D models. This data can be used to analyze and represent information related to emotions, feelings, communication, and social interactions.

[0026] By controlling the data through the interaction between the synchronous devices of the vehicle terminal, the virtual initial model of the vehicle robot is tightly connected with the interactive terminal, thereby obtaining motion sequence data and facial animation data based on the import of the virtual initial model.

[0027] Step S20: Respond to the operation command of the interactive terminal, adjust the action sequence data and / or facial animation data according to the operation command to obtain target data information.

[0028] It's important to note that an interactive terminal typically refers to a user interface within a computer system, where users can interact with the computer by entering text commands without needing a graphical user interface (GUI). In Unix-like systems (such as Linux), this interactive terminal is often called a "terminal," "console," or "command-line interface." An operation command is a way to instruct the computer to perform a specific task or operation by entering a specific text string in a computer terminal or command-line interface. Operation commands are a way for the computer's operating system and applications to interact; they are used to control various functions of the computer system, from file management and system configuration to network connections. An operation command typically consists of a command keyword, sometimes accompanied by arguments and options that tell the computer how to execute the command. The command keyword is the name of the operation to be performed, while the arguments and options specify the specific object, settings, or method of the operation. Operation commands can include various forms such as mouse dragging, touch screen swiping, and button clicking. Each type of command will be mapped to the corresponding motion sequence data and facial animation data adjustment rules. Regarding the adjustment of motion sequence data, if the designer requires the onboard robot to turn in a certain direction or perform a specific action, the program will modify the motion trajectory and rotation angle of the onboard robot in the virtual initial model according to the command.

[0029] It should be understood that a vehicle-mounted robot is a type of intelligent robot with the ability to mimic the appearance and movements of a human head. Employing advanced sensors and computing technology, it can perceive and interpret information from the external world and react accordingly with appropriate actions and expressions. Furthermore, the movements of a vehicle-mounted robot encompass multiple aspects, including but not limited to head rotation, eye movement, lip expressions, and facial muscle contractions. These movements can be used individually or in combination, enabling the robot to simulate a rich variety of expressions and modes of communication, such as smiling, blinking, nodding, shaking its head, and frowning.

[0030] Designers adjust the motion sequence data and / or facial animation data of the onboard robot's head in real time via operation commands. The motion design program first parses and processes the received operation commands to identify the designer's intentions and requirements. For example, if the designer instructs the robot to turn its head to the left, the motion design program will adjust the robot's steering angle accordingly to turn left and then generate new motion sequence data. At this point, the motion design program will adjust the shape of the robot's mouth and the state of its eyes to make it display the expression of turning its head to the left, and generate corresponding facial animation data. The adjusted motion sequence data and / or facial animation data of the onboard robot's head, which is the target data information, is obtained.

[0031] Step S30: The target data information is sent to the vehicle-mounted display terminal through the communication module, so that the vehicle-mounted display terminal can synchronously display actions and / or expressions according to the target data information.

[0032] It's important to note that a communication module is a device, component, or system used to transmit information, data, or signals between different devices to achieve information exchange and data transfer. Communication modules can employ various technologies and protocols, and depending on the application requirements, they can be used for wired or wireless communication. The function of a communication module is to transmit data from one device to another; it can be a computer, sensor, controller, mobile device, etc. Communication modules play a crucial role in modern technology, enabling various devices to communicate and coordinate their operations. An in-vehicle display terminal refers to a device or system installed inside a vehicle to provide various information, entertainment content, or functions to passengers or the driver. These terminals typically feature a display screen (which may be a touchscreen), audio output, and a user interface, allowing users to interact with them.

[0033] It should be understood that during data transmission, the communication module transmits the adjusted target data information to the vehicle-mounted display terminal in an efficient and reliable manner. This typically involves technologies such as data encoding, compression, and encryption to ensure the security and reliability of data transmission. Once the vehicle-mounted display terminal receives the target data information, it immediately uses this data to synchronously display the vehicle-mounted robot's movements and expressions. The vehicle-mounted display terminal may be equipped with a high-resolution display screen, as well as a flexible motion control system and expression display device. These hardware devices allow the vehicle-mounted robot to display various movements and expressions in a detailed and vivid manner on the vehicle-mounted display terminal.

[0034] By interacting in real time with the communication module, the vehicle-mounted display terminal can accurately capture the motion sequences and facial animations in the target data, and then dynamically adjust the vehicle-mounted robot based on the data. This means that when a designer issues an operation command on the interactive terminal, the vehicle-mounted display terminal can react instantly and synchronously display the vehicle-mounted robot's movements and expressions.

[0035] In this embodiment, the above-described scheme is implemented by acquiring an imported virtual initial model, obtaining action sequence data and facial animation data based on the virtual initial model, responding to the operation instructions of the interactive terminal, and adjusting the action sequence data and / or facial animation data according to the operation instructions to obtain target data information. The target data information is sent to the vehicle-mounted display terminal via a communication module, enabling the vehicle-mounted display terminal to synchronously display actions and / or expressions based on the target data information. This allows for the application of operational commands to the virtual initial model, while simultaneously transmitting data information to the vehicle-mounted robot, achieving rapid verification of the action design and enabling real-time synchronous comparison of the vehicle-mounted robot's actions with the actual robot's movements, thus improving design efficiency.

[0036] Reference Figure 2 Based on the above embodiments, it should be noted that, firstly, the accurate 3D model of the vehicle-mounted robot's head needs to be imported into dedicated design tools and plugins, which are typically embedded in a personal computer (PC) environment. Once successfully imported, these tools and plugins will begin processing this virtual initial model, i.e., the 3D model. During processing, the plugins will analyze motion sequence data and facial animation data, which describe the vehicle-mounted robot's head's responses and facial expressions in different situations. These plugins will establish a connection with Simbox using serial port technology to ensure smooth data transmission. Simbox acts as a bridge for data transmission, responsible for organizing and packaging the data obtained from the design tools and plugins. Subsequently, via the Controller Area Network (CAN) protocol, data will be transmitted from Simbox to the vehicle-mounted robot head system (an embedded product installed inside a car, capable of communicating with the car's CAN network, and shaped like a head). The CAN protocol, as a highly efficient and reliable communication protocol, ensures rapid data transmission within the vehicle system, thereby driving the vehicle-mounted robot head to perform corresponding actions and facial expressions.

[0037] Reference Figure 3 Based on the above embodiments, in the second embodiment, step S20 includes: Step S210: When the operation instruction is a drag instruction, the drag instruction is parsed to obtain the operation information of the virtual initial model.

[0038] It should be understood that drag-and-drop commands typically refer to the operation of moving an element from one location to another in a computer interface using an input device such as a mouse or touchscreen. This operation is commonly used in graphical user interfaces (GUIs) for moving, copying, rearranging, or manipulating objects, files, icons, etc., on the interface. Designers can perform gesture-based drag-and-drop operations on interactive terminals, such as dragging a virtual model of a vehicle-mounted robot with their finger. By parsing drag-and-drop commands, motion design programs can accurately capture the designer's operational intentions, enabling precise control of the vehicle-mounted robot on the interactive terminal, thereby displaying the initial virtual model.

[0039] Step S220: Adjust the action sequence data and / or facial animation data according to the operation information to obtain target data information.

[0040] When the interactive terminal receives a drag command, the running motion design program parses and processes the command to obtain relevant drag information. First, the interactive terminal captures the drag trajectory and speed of the designer's gesture on the screen. Then, the motion design program analyzes the operation of the virtual initial model based on this information, including the rotation direction, angle, and drag distance of the vehicle-mounted robot. During the parsing of the drag command, the motion design program may also dynamically adjust based on the vehicle's motion state and environmental information.

[0041] By parsing drag-and-drop commands and acquiring operational information, the interactive terminal can achieve real-time interactive control with the vehicle-mounted robot. Designers can freely control the vehicle-mounted robot's movements on the interactive terminal through simple and intuitive gesture drag-and-drop operations, thereby obtaining target data information.

[0042] This embodiment, through the above-described scheme, specifically by parsing the drag command when the operation instruction is a drag command, obtains the operation information of the virtual initial model, and adjusts the action sequence data and / or facial animation data according to the operation information to obtain the target data information. Thus, by parsing the instruction, the virtual initial model can be quickly obtained.

[0043] Reference Figure 4 Based on the above embodiments, in the third embodiment, adjusting the action sequence data according to the operation instruction to obtain target data information includes: Step S310: Adjust the action sequence data according to the operation instruction to obtain the adjusted action sequence data.

[0044] It should be understood that the motion design program adjusts the motion sequence data based on the motion range and possible motion sequences of the onboard robot in the virtual initial model, resulting in the adjusted motion sequence data.

[0045] Step S320: Based on the correspondence between action sequence data and facial expression animation data, determine the first facial expression animation data corresponding to the adjusted action sequence data.

[0046] It's important to note that the initial model data is analyzed to understand the relationship between motion sequences and facial animations. Machine learning algorithms or statistical methods are then used to construct a mapping model between the motion sequence data and the facial animation data. The target motion sequence data is input into this mapping model to obtain the corresponding facial animation data. If multiple samples of the target motion sequence data exist, they need to be mapped one by one. Based on the results obtained from the mapping model, the corresponding first facial animation data is generated. This facial animation data can be a series of keyframes or a continuous animation sequence, depending on the characteristics of the mapping model and the original data.

[0047] Step S330: Obtain target data information based on the adjusted action sequence data and the first facial expression animation data.

[0048] It should be understood that when both the adjusted motion sequence data and the first facial expression animation data exist, the target data information is obtained.

[0049] This embodiment, through the above-described scheme, specifically obtains adjusted action sequence data by adjusting the action sequence data according to operation instructions; determines the first facial expression animation data corresponding to the adjusted action sequence data based on the correspondence between the action sequence data and facial expression animation data; and obtains target data information based on the adjusted action sequence data and the first facial expression animation data. This allows for more convenient adjustment of facial expressions through action sequences.

[0050] Reference Figure 5 Based on the above embodiments, in the fourth embodiment, before determining the first facial animation data corresponding to the adjusted action sequence data, the following steps are included: Step S410: Match the adjusted action sequence data with the standard action sequence data.

[0051] It's important to note that standard action sequence data typically refers to a predefined set of operational steps within a specific domain or task, used to perform a particular task or achieve a specific goal. These sequences are considered standard ways to complete a task in a specific context and can be used for training, automation, process optimization, and other purposes. Preprocessing is performed on the adjusted action sequence data and the standard action sequence data to ensure data quality and consistency. Possible preprocessing steps include data cleaning, missing value imputation, and data normalization. Features are then extracted from the adjusted action sequence data and the standard action sequence data. Feature extraction aims to represent the data in a more informative form for subsequent matching calculations.

[0052] Appropriate similarity metrics are used to compare the similarity between feature representations. Common similarity metrics include Euclidean distance, cosine similarity, and correlation coefficient. Based on the similarity metrics, matching algorithms are applied to determine whether the adjusted action sequence data matches the standard action sequence data. Matching algorithms can be rule-based or machine learning-based, such as k-nearest neighbor and support vector machine algorithms.

[0053] A similarity threshold is set to determine whether two sequence data match. Based on the matching algorithm and the similarity threshold, it is determined whether the adjusted action sequence data matches the standard action sequence data. If the similarity measure exceeds the preset threshold, the two sequences are considered to be successfully matched; otherwise, the match is considered to have failed.

[0054] Step S420: If the adjusted action sequence data matches the standard action sequence data, then proceed to step S330.

[0055] If a match is successful, subsequent steps can be performed. Based on the matched standard action sequence data, the most suitable template is selected from a predefined facial animation template library. Each template in the library represents a different expression and emotional state, so selecting the template with the highest degree of matching to the action sequence data can better represent the consistency between action and emotion. This generates the first facial animation data.

[0056] Optionally, the generated first facial expression animation data can be visualized, allowing users or operators to observe the animation effects in real time. At this stage, users can provide feedback, such as adjusting parameters or modifying template selection, to further optimize the animation quality. The generated first facial expression animation data is then carefully checked and verified to ensure consistency between the animation and the original motion sequence data, and to evaluate the quality and conformity of the facial expression animation.

[0057] Step S430: If the adjusted action sequence data does not match the standard action sequence data, an error message is displayed through the interactive terminal.

[0058] If the matching fails, exception handling is required. During the matching process, if the adjusted action sequence data is found to be mismatched with the standard action sequence data, the system will automatically trigger the exception detection mechanism. The system will issue an exception message to the user or operator through an interactive terminal, such as a graphical interface or command-line interface. This includes a clear indication of matching failure and may provide some related prompts, such as which parts are mismatched and possible reasons.

[0059] It's important to note that after receiving an anomaly notification, the user or operator needs to confirm whether they understand the error message. In some cases, the user may need to provide further feedback or choose whether to continue processing. Based on the anomaly notification, the user or operator may need to check and correct the adjusted motion sequence data. This includes possible data cleaning, correction, or parameter readjustment. After data correction, the system may re-run the matching process to verify whether the anomaly has been successfully resolved. If the matching problem is resolved, the step of generating the first facial animation data can continue. If the matching problem persists, the system will trigger the anomaly notification again, entering a feedback loop until a successful match is achieved. In some cases, the matching problem may be complex and require manual intervention. In such cases, the system may provide a corresponding interface or guidance to allow the user or operator to participate in the matching process and offer options for manually adjusting or annotating the data.

[0060] This embodiment, through the above-described scheme, specifically matches the adjusted action sequence data with the standard action sequence data; if the adjusted action sequence data matches the standard action sequence data, the step of executing the first facial expression animation data corresponding to the adjusted action sequence data is performed; if the adjusted action sequence data does not match the standard action sequence data, an error message is displayed through the interactive terminal. Thus, based on the above operation matching judgment, the rationality of the user's or operator's action design is ensured.

[0061] Reference Figure 6 Based on the above embodiments, in the fifth embodiment, after step S30, the following steps are included: Step S510: Render the action and / or expression, and stop the interaction when the rendered action and / or expression are displayed synchronously.

[0062] It should be understood that rendering the generated actions and / or expressions involves converting the action sequence into consecutive animation frames, applying facial expression data and emotional states, changing the model's pose and expression, and adding scene and lighting effects. These steps ultimately produce realistic facial animation effects. The rendered actions and / or expressions are then synchronously displayed on an interactive terminal. This could be a graphical interface or a video player to display the generated facial animations. Interactive functionality is disabled while the actions and / or expressions are being synchronously displayed.

[0063] This embodiment, through the above-described solution, specifically by rendering actions and / or expressions, stops the interaction when the rendered actions and / or expressions are simultaneously displayed. This allows the interaction to end, ensuring the continuity of the function.

[0064] Reference Figure 7Based on the above embodiments, it should be noted that the data layering is divided into a data layer, a business layer, and a device layer. The data layer includes Resources, JSON files, and H264 files. The business layer includes resource loading, animation editing, animation preview, serial communication, animation saving, image rendering, and video compositing. The device layer includes Simbox and NomiMate. Resources from the data layer are passed to the business layer, which then passes them to the Simbox in the device layer. The Simbox passes them to NomiMate and also sends them back to the business layer, which then sends the JSON and H264 files back to the data layer.

[0065] Reference Figure 8 Based on the above embodiments, the sixth embodiment further includes: Step S610: Receive the data information, convert the data information into a data stream, implement the corresponding caching strategy, and transmit it.

[0066] It's important to note that data stream caching strategies refer to a set of rules and methods used to manage data streams in a computer system. Data stream caching strategies aim to optimize data reading and writing to improve system performance, reduce latency, and effectively utilize memory and storage resources. Data streams, on the other hand, are the concept of data flowing and being transmitted within a computer system; they can include communication between internal components, communication over a network, and read / write operations on storage media.

[0067] It should be understood that converting data information into a data stream refers to transforming data from its original representation or form into a form that can flow and be transmitted within a computer system for processing, storage, analysis, or transmission. This conversion typically involves encoding the data into a format suitable for transmission and may include data compression, encryption, or other processing steps. Then, an appropriate caching strategy is implemented. The caching strategy determines which data streams will be cached, as well as the cache size, replacement strategy, etc. This helps improve access speed and system performance. The data stream managed by the caching strategy is then transmitted over the network or within the system. This may involve packet segmentation, selection of transport protocols, and transmission control. The transmitted data stream is received at the destination location (e.g., another computer, device, or system component). If the receiving end also has a cache, then a cache management strategy also needs to be implemented at the receiving end to ensure that the data is properly processed and stored at the receiving end.

[0068] Optionally, the data interaction sequence between the design tool plugin and Simbox is described as follows: At the start of the interaction, after a theoretical delay (40ms), head rotation information and camera stream data are cached for the first time. After a period of time (e.g., 1 second), the ffmpeg command is used to save the stream data as an h264 file, and the first video playback message and head rotation message are sent. After another theoretical delay (40ms), head rotation messages are sent intermittently, and head rotation information and camera stream data are cached. After another period of time (e.g., 1 second), the ffmpeg command is used to save the stream data as an h264 file, the video playback message is sent, and the sending of head rotation information begins, consistent with the previously described steps. This continues until the interaction stops, with the final sending of the video playback message and the final sending of the head rotation message.

[0069] This embodiment, through the above-described scheme, specifically by receiving the data information, converting it into a data stream, implementing an appropriate caching strategy, and then transmitting it, ensures that the data is properly processed and stored during transmission.

[0070] Reference Figure 9 Based on the above embodiments, it should be noted that after the designer performs the first operation, they will choose whether to import the design tool plugin to load the robot head model scene, and then they will make a judgment on whether to select real-time interaction.

[0071] If selected, real-time interaction begins. Interaction starts via a Simbox connection. Left-clicking and sliding the mouse sends real-time head rotation information messages. The mouse movement is saved as an action sequence, and the robot displays the head movements in real-time. Simultaneously, a matching facial expression animation plays. Each frame caches real-time camera rendering data to create an h264 file, and plays the h264 file in real-time, allowing the robot to display expressions synchronously. The head model can also be controlled using a plugin during interaction, with the design tool plugin simultaneously displaying the actions and animations.

[0072] If no option is selected, the motion sequence will be replayed, loading the pre-designed head motion sequence and accompanying facial expression animation. The motion sequence and animation will then be played synchronously. The plugin connects to Simbox, transmitting the head motion sequence and expressions in real-time according to the pause rate. The robot displays the head motion and expression animations in real-time. The motion sequence and animation are played synchronously, and the head model is controlled synchronously using the plugin. The design tool plugin synchronously displays the motion and animation.

[0073] Reference Figure 10Based on the above embodiments, in the seventh embodiment, the vehicle terminal synchronization system includes an interactive terminal 10, a communication module 20, and a vehicle display terminal 30. The interactive terminal 10 runs an action design program, and the communication module 20 enables the interactive terminal 10 to interact with the vehicle display terminal 30.

[0074] It should be noted that the interactive terminal 10 is used to acquire the imported virtual initial model, obtain action sequence data and facial animation data based on the virtual initial model; to respond to the operation instructions of the interactive terminal 10, adjust the action sequence data and / or facial animation data according to the operation instructions to obtain target data information; and to send the target data information to the vehicle display terminal 30 through the communication module 20 so that the vehicle display terminal can synchronously display actions and / or expressions according to the target data information.

[0075] It should be understood that the interactive terminal 10 is also used to store head motion sequences and facial expression animations, call the motion design program, call the head motion sequences and facial expression animations through the motion design program, and send the head motion sequences and facial expression animations to the vehicle display terminal 30 for playback.

[0076] In this embodiment, the interactive terminal 10 is connected to the communication module 20 via a serial port, and the communication module 20 is connected to the vehicle-mounted display terminal 30 via a CAN bus. The communication module 20 is used to transmit target data information transparently to the vehicle-mounted display terminal 30.

[0077] This embodiment uses a vehicle-mounted terminal synchronization system, which includes an interactive terminal 10, a communication module 20, and a vehicle-mounted display terminal 30, to easily and quickly synchronize and compare the movement effects of the vehicle-mounted robot in real time.

[0078] In one embodiment, the interactive terminal 10 is further configured to store head motion sequences and facial expression animations, call a motion design program, invoke the head motion sequences and facial expression animations through the motion design program, and send the head motion sequences and facial expression animations to the vehicle-mounted display terminal 30 for playback.

[0079] In one embodiment, the interactive terminal is connected to the communication module 20 via a serial port, and the communication module 20 is connected to the vehicle-mounted display terminal 30 via a CAN bus. The communication module 20 is used to transmit the target data information to the vehicle-mounted display terminal.

[0080] In one embodiment, the communication module 20 is further configured to convert the target data information into a data format supported by the vehicle-mounted display terminal 30, and send the converted target data information to the vehicle-mounted display terminal 30.

[0081] In addition, to achieve the above objectives, the present invention also proposes an interactive terminal, which is used to store head motion sequences and facial expression animations, call a motion design program, call the head motion sequences and facial expression animations through the motion design program, and send the head motion sequences and facial expression animations to the vehicle display terminal for playback.

[0082] Since the present invention adopts all the technical solutions of all the above embodiments, it has at least all the beneficial effects brought about by the technical solutions of the above embodiments, which will not be repeated here.

[0083] It should be understood that the above are merely illustrative examples and do not constitute any limitation on the technical solution of the present invention. In specific applications, those skilled in the art can make settings as needed, and the present invention does not impose any restrictions on this.

[0084] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0085] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0086] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0087] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A synchronization method for an in-vehicle terminal, characterized in that, The vehicle-mounted terminal synchronization method is applied to a vehicle-mounted terminal synchronization system. The vehicle-mounted terminal synchronization system includes an interactive terminal, a communication module, and a vehicle-mounted display terminal. The interactive terminal runs an action design program, and the communication module enables information interaction between the interactive terminal and the vehicle-mounted display terminal. The vehicle-mounted terminal synchronization method includes: Obtain the imported virtual initial model, and obtain action sequence data and facial animation data based on the virtual initial model; In response to the operation command of the interactive terminal, the action sequence data and / or facial animation data are adjusted according to the operation command to obtain target data information; The step of responding to the operation command of the interactive terminal and adjusting the action sequence data and / or facial animation data according to the operation command to obtain target data information includes: When the operation instruction is a drag instruction, the drag instruction is parsed to obtain the operation information of the virtual initial model; Adjust the action sequence data and / or facial animation data according to the operation information to obtain target data information; The process of adjusting the action sequence data according to the operation instructions to obtain target data information includes: The action sequence data is adjusted according to the operation instructions to obtain the adjusted action sequence data; Based on the correspondence between action sequence data and facial expression animation data, determine the first facial expression animation data corresponding to the adjusted action sequence data; Based on the adjusted action sequence data and the first facial expression animation data, target data information is obtained; The target data information is sent to the vehicle-mounted display terminal through the communication module, so that the vehicle-mounted display terminal can synchronously display actions and / or expressions based on the target data information.

2. The synchronization method for the vehicle-mounted terminal as described in claim 1, characterized in that, Before determining the first facial animation data corresponding to the adjusted action sequence data, the following steps are included: The adjusted action sequence data is matched with the standard action sequence data; If the adjusted motion sequence data matches the standard motion sequence data, then the step of executing the first facial animation data corresponding to the adjusted motion sequence data is performed; If the adjusted action sequence data does not match the standard action sequence data, an error message will be displayed through the interactive terminal.

3. The synchronization method for the vehicle-mounted terminal as described in claim 1, characterized in that, After the step of sending the target data information to the vehicle-mounted display terminal via the communication module, so that the vehicle-mounted display terminal can synchronously display actions and / or expressions according to the target data information, the method further includes: The actions and / or expressions are rendered, and the interaction stops when the rendered actions and / or expressions are displayed synchronously.

4. The synchronization method for the vehicle-mounted terminal as described in any one of claims 1 to 3, characterized in that, Also includes: The system receives the data information, converts it into a data stream, implements a corresponding caching strategy, and then transmits it.

5. A synchronization system for an in-vehicle terminal, characterized in that, The synchronization system of the vehicle terminal includes an interactive terminal, a communication module, and a vehicle display terminal. The interactive terminal runs an action design program, and the communication module enables the interactive terminal to interact with the vehicle display terminal. The interactive terminal is used to acquire the imported virtual initial model and obtain action sequence data and facial animation data based on the virtual initial model. The interactive terminal is used to respond to the operation instructions of the interactive terminal, and adjust the action sequence data and / or facial animation data according to the operation instructions to obtain target data information; The interactive terminal is configured to respond to operation commands from the interactive terminal and adjust the motion sequence data and / or facial animation data according to the operation commands to obtain target data information. This includes: when the operation command is a drag command, parsing the drag command to obtain operation information of the virtual initial model; adjusting the motion sequence data and / or facial animation data according to the operation information to obtain target data information; wherein adjusting the motion sequence data according to the operation commands to obtain target data information includes: adjusting the motion sequence data according to the operation commands to obtain adjusted motion sequence data; determining the first facial animation data corresponding to the adjusted motion sequence data according to the correspondence between motion sequence data and facial animation data; and obtaining target data information based on the adjusted motion sequence data and the first facial animation data. The interactive terminal is used to send the target data information to the vehicle-mounted display terminal through the communication module, so that the vehicle-mounted display terminal can synchronously display actions and / or expressions according to the target data information.

6. The synchronization system for the vehicle-mounted terminal as described in claim 5, characterized in that, The interactive terminal is also used to store head motion sequences and facial expression animations, call the motion design program, call the head motion sequences and facial expression animations through the motion design program, and send the head motion sequences and facial expression animations to the vehicle terminal for playback.

7. The synchronization system for the vehicle-mounted terminal as described in claim 5, characterized in that, The interactive terminal is connected to the communication module via a serial port, and the communication module is connected to the vehicle-mounted display terminal via a CAN bus. The communication module is used to transmit the target data information to the vehicle-mounted display terminal.

8. The synchronization system for the vehicle-mounted terminal as described in claim 7, characterized in that, The communication module is also used to convert the target data information into a data format supported by the vehicle-mounted display terminal, and send the converted target data information to the vehicle-mounted display terminal.

Images