Mobile terminal digital twin model construction method and system

Abstract

The invention discloses a method and a system for constructing a digital twin model of a mobile terminal. The method comprises the following steps: acquiring non-relational multi-source heterogeneous mobile terminal data; constructing a three-dimensional memory mapping relation model to obtain a mobile terminal database; a data retrieval service is provided for the mobile terminal database, and data retrieval of vectorized data in the mobile terminal database is achieved; constructing a virtual entity mapping space according to vectorized data of data retrieval; and defining an application agent, and obtaining a mobile terminal digital twinborn model for realizing user environment real-time sensing, processing and executing RPA according to the virtual entity mapping space. The problems that non-relational multi-source heterogeneous mobile terminal data fragment information cannot be effectively collected and integrated and multi-source heterogeneous mobile terminal data cannot be subjected to structured storage in an existing information technology are solved. And meanwhile, technical support cannot be provided for the user in the aspects of managing own data security, maintaining data rights and interests, processing and using data, knowing data conditions and the like.

Description

Technical Field

[0001] The present invention relates to the technical field of digital twins, and in particular, to a method and system for constructing a digital twin model of a mobile terminal. Background Art

[0002] With the continuous progress of big data and artificial intelligence technologies, the core value of data has become increasingly important in the wave of new-generation information technologies. Building a reliable data ecosystem and maximizing the value of data elements have become the key guidelines for promoting future data development. In this process, not only large enterprises and institutions can generate intuitive and highly valuable data, but individual users also generate a large amount of scattered data resources in their daily activities. However, the potential value of this data is often overlooked. This data can widely and deeply map various aspects of users' lives, from daily consumption habits, social interactions to work and study situations. It not only includes users' basic information such as age, gender, and geographical location, but also covers richer details such as users' hobbies, emotional states, psychological activities, health conditions, and users' behavior patterns and decision-making processes in different scenarios. The data generated by users in their daily lives is an important window for understanding their lifestyles, demand preferences, and potential behavior trends.

[0003] The current service mode of information technology mainly relies on manufacturers. A large amount of data generated and involved by individual users in their daily use, including personal information, usage records, preference settings, etc., are almost all stored and saved on the servers or data platforms of these manufacturers. In addition, data between major application service manufacturers cannot be interconnected and interoperated, a complete data export service cannot be provided to users, and the use of data cannot be effectively converted into the actual rights and interests of users, resulting in great difficulties and challenges for users in managing their data security, safeguarding data rights and interests, processing and using data, and understanding the data situation. Since there is currently no powerful way to collect and integrate these fragmented information, the value of personal life data cannot be fully and effectively released. How to effectively collect personal data and convert it into more valuable information for use and circulation has become an urgent problem to be solved.

[0004] While releasing the value of user data and ensuring user data security, data storage and use need to be completed on the local mobile terminal. The existing technical solutions currently have the following problems: First, the data types are diverse and the sources are extensive. It is difficult to effectively expand and utilize multi-source heterogeneous data with traditional single-table storage or JSON format storage methods. Second, when using the large model RAG technology, due to the huge word embedding model, the mobile terminal cannot run, so the TF-IDF or BM25 algorithm is mostly used, resulting in semantic loss and affecting the data utilization effect. Third, how to use artificial intelligence technology to ensure user data security while bringing value feedback to users, reducing the complexity of using technologies such as AIGC, LLM, and AGENT, and building a complete personal digital twin space. Summary of the Invention

[0005] The present invention provides a method and system for constructing a mobile terminal digital twin model to overcome the above technical problems.

[0006] To achieve the above object, the technical solution of the present invention is: A method for constructing a mobile terminal digital twin model includes the following steps: S1: Collect and obtain non-relational multi-source heterogeneous mobile terminal data; And the multi-source heterogeneous mobile terminal data includes mobile terminal application data, hardware sensor data, and user private data; S2: Construct a three-dimensional memory mapping relationship model; And store the multi-source heterogeneous mobile terminal data based on the three-dimensional memory mapping relationship model to obtain a three-dimensional memory mapping relationship table for storing non-relational data, and redefine the field names of the multi-source heterogeneous mobile terminal data as the field indexes corresponding to the non-relational mobile terminal data fields through the three-dimensional memory mapping table; Perform vectorization processing on the multi-source heterogeneous mobile terminal data obtained through the field indexes to obtain vectorized data for storage and obtain a mobile terminal database; S3: Provide data retrieval services for the mobile terminal database to implement data retrieval operations on the vectorized data in the mobile terminal database; S4: Construct a virtual entity mapping space according to the vectorized data obtained from the data retrieval operation; Define an application intelligent agent applied to the virtual entity mapping space, and obtain a mobile terminal digital twin model for realizing real-time perception processing and execution of RPA in the user environment according to the virtual entity mapping space.

[0007] Further, the S2 specifically includes the following steps: S21: Construct a three-dimensional memory mapping relationship model, which includes a data table setting module, a field set acquisition module, a data type specification module, and a three-dimensional memory mapping relationship module; The data table setting module is used to set the data table named data for storing multi-source heterogeneous mobile terminal data according to the set SQLite database; The field set acquisition module is used to add data fields to the multi-source heterogeneous mobile terminal data stored in the data table to obtain the data field set F; And F = {data_0, data_1, …, data_n}; where data_n represents the nth added data field; The data type specification module is used to set the type label set of each data field type in the data field set; The three-dimensional memory mapping relationship module is used to construct a three-dimensional memory mapping relational expression according to the data field set and the type label set, and obtain a three-dimensional memory mapping table through the three-dimensional memory mapping relational expression; And the three-dimensional memory mapping relational expression is M: F × Ttype × N → V; where M represents the abbreviated form of M(f, t, n); Ttype represents the type label set; N represents the unique identifier of the data record; V represents the data value containing all the multi-source heterogeneous mobile terminal data to be stored; f represents the data field and f ∈ F; t represents the data type and t ∈ Ttype; n represents the value corresponding to the record identifier and n ∈ N; S22: Redefine the field names of the multi-source heterogeneous mobile terminal data as the field indexes corresponding to the non-relational mobile terminal data fields through the obtained three-dimensional memory mapping table; S23: Based on the pre-trained continuous bag-of-words model CBOW, perform vectorization processing on the multi-source heterogeneous mobile terminal data obtained through the field indexes, obtain vectorized data for storage, and obtain the mobile terminal database.

[0008] Furthermore, the data retrieval service in S3 includes a data retrieval service based on semantic retrieval and a data retrieval service based on SQL retrieval; The retrieval method of the data retrieval service based on semantic retrieval: Based on the mobile terminal database, construct several data vector index navigation layer nodes for data retrieval through the HNSW graph algorithm; And obtain the edge weights of the data vector index navigation layer nodes based on the heuristic search method; Obtain the index navigation graph structure for data retrieval according to the edge weights and the data vector index navigation layer nodes; Based on the embedded vector search engine J Vector of single instruction multiple data streams SIMD, implement data indexing of the mobile terminal database according to the index navigation graph structure; The retrieval method of the data retrieval service based on SQL retrieval: Semantically parse the data in the mobile terminal database through the large language model LLM to obtain a structured query intention representation; Based on the pre-trained Text2SQL model, obtain an SQL statement according to the structured query intention representation; Through the local data storage method technology, based on the SQL statement, obtain an SQL query statement according to the storage format of the local database and the field index characteristics of the vectorized data, so as to realize the data indexing of the mobile terminal database according to the SQL query statement.

[0009] Further, step S4 specifically includes the following steps: S41: Customize the entity model and entity events of any mobile terminal; Customize and construct a Prompt text for obtaining a structured semantic description framework according to the entity model and entity events of the mobile terminal through natural language technology; S42: Invoke the AIGC model of text-to-image / video-to-video, and obtain a virtual digital entity model representation according to the Prompt text; S43: Convert the Prompt text into a query vector to be retrieved, and based on the data retrieval service provided by the mobile terminal database, retrieve and obtain multi-source heterogeneous mobile terminal data related to the virtual digital entity model representation through the query vector to be retrieved; And in combination with the pre-trained large language model LLM, obtain an overview report of entity events according to the retrieved multi-source heterogeneous mobile terminal data, and the overview report of entity events is the virtual entity mapping space where the user is located; S44: Define an application agent applied to the virtual entity mapping space, and based on the overview report of entity events retrieved and obtained from the virtual entity mapping space, determine the execution action instruction for a certain scenario in the current user environment; S45: Perform the action execution of the mobile terminal according to the execution action instruction to realize the function of real-time perception processing and execution of RPA in the user environment, and further realize the construction of the mobile terminal digital twin model; And the execution action instruction at least includes an operation instruction for the user's mobile terminal interface and an access instruction for the remote server interface.

[0010] Further, the method for collecting and obtaining mobile terminal application data in S1 includes an active application data collection method and a passive application data collection method based on the accessibility service; The active application data collection method: S001: Enable the accessibility service permission of the Android system: Including enabling the permission setting for allowing gesture operation to be used for the simulated click operation of the mobile terminal; Enable the permission setting for allowing traversal of window content to obtain interface data; S002: Open a floating window for collecting application data of the mobile terminal; S003: Call the background server to download an application data collection plugin to collect and obtain structured data of the mobile terminal's application data based on simulated click operations according to the position of the floating window; The passive application data collection method: S100: Enable the permission of the Android system's accessibility service: Include enabling the permission setting for allowing gesture operations to be executed for simulated click operations on the mobile terminal; Enable the permission setting for allowing traversal of window content to obtain interface data; S101: Listen for changes in the mobile terminal page in real time by calling the accessibility service event listening tool, and when the page changes, collect the structured data of the real-time display of the mobile terminal page by calling the interface activity window node acquisition tool; otherwise, do not perform the collection action; The method of listening for changes in the mobile terminal page in real time by calling the accessibility service event listening tool: S1011: Obtain the click moment when the mobile terminal receives an interface click event, and use it as the first click moment; Obtain the second click moment when the mobile terminal receives an interface click event, and the second click moment is the click moment corresponding to the interface click event after the interface click event corresponding to the first click moment; S1012: Obtain the click time interval between the second click moment and the first click moment; Judge the size between the click time interval and the preset time interval threshold; If the click time interval is greater than the preset time interval threshold, confirm that the mobile terminal page has changed; if the click time interval is less than or equal to the preset time interval threshold, confirm that the mobile terminal page has not changed.

[0011] Furthermore, S1 also includes a method for data compression of multi-source heterogeneous mobile terminal data: Encode the source symbols in the multi-source heterogeneous mobile terminal data based on Huffman coding to obtain source coded symbols; Obtain the occurrence probability of each source coded symbol, and perform data compression on the multi-source heterogeneous mobile terminal data based on the probability statistical model of the source according to the source coded symbols; And the expression of the probability statistical model based on the source is , In the formula: Represents the expected length of data compression; represents the average code length; represents the coding length; E represents the expectation; represents the description of summing over all possible source coding symbols Ξ represents the set of source coding symbols; represents the source coding symbol the probability of occurrence; represents the source coding symbol the corresponding coding length.

[0012] A mobile terminal digital twin model construction system includes a data acquisition module, a data compression module, a three-dimensional memory mapping relationship model construction module, a data vectorization module, a data retrieval service module, an application agent setting module, and a mobile terminal digital twin model construction module; The data acquisition module is used to collect and obtain non-relational multi-source heterogeneous mobile terminal data; And the multi-source heterogeneous mobile terminal data includes mobile terminal application data, hardware sensor data, and user data; The data compression module is used to compress the multi-source heterogeneous mobile terminal data to obtain compressed data; The three-dimensional memory mapping relationship model construction module is used to construct a three-dimensional memory mapping relationship model; And based on the three-dimensional memory mapping relationship model, store the multi-source heterogeneous mobile terminal data to obtain a three-dimensional memory mapping relationship table for storing non-relational data, and redefine the field names of the multi-source heterogeneous mobile terminal data as the field indexes corresponding to the non-relational mobile terminal data fields through the three-dimensional memory mapping table; The data vectorization module is used to perform vectorization processing on the multi-source heterogeneous mobile terminal data obtained through the field index, obtain vectorized data for storage, and obtain a mobile terminal database; The data retrieval service module is used to provide data retrieval services for the mobile terminal database to implement data retrieval operations on the vectorized data in the mobile terminal database; And the data retrieval service includes a data retrieval service based on semantic retrieval and a data retrieval service based on SQL retrieval; The application agent setting module is used to set application agents applied to the virtual entity mapping space, and the application agents at least include an information security agent and a health management agent; The mobile terminal digital twin model construction module is used to construct a virtual entity mapping space according to the vectorized data obtained from the data retrieval operation, and based on the application agent, obtain a mobile terminal digital twin model for realizing real-time perception processing and execution of RPA in the user environment according to the virtual entity mapping space.

[0013] Beneficial effects: The present invention provides a method and system for constructing a digital twin model of a mobile terminal. By storing multi-source heterogeneous mobile terminal data based on the constructed three-dimensional memory mapping relationship model, a three-dimensional memory mapping relationship table for storing non-relational data is obtained, effectively realizing the acquisition and integration of non-relational fragmented information of multi-source heterogeneous mobile terminal data, and greatly improving the comprehensiveness of obtaining the digital information of the user's mobile terminal. By redefining the field names of multi-source heterogeneous mobile terminal data as field indexes corresponding to non-relational mobile terminal data fields through the three-dimensional memory mapping table, and performing vectorization processing on the multi-source heterogeneous mobile terminal data obtained through the field indexes to obtain vectorized data for storing and obtaining a mobile terminal database, it effectively realizes the structured storage of multi-source heterogeneous mobile terminal data, laying a foundation for providing users with a complete data export service. By defining an application agent applied to the virtual entity mapping space and obtaining the construction of a digital twin model of a mobile terminal for realizing real-time perception processing and execution of RPA in the user environment according to the constructed virtual entity mapping space, it provides technical support for users in aspects such as managing their own data security, maintaining data rights and interests, processing and using data, and understanding the data status, greatly improving the management and application of mobile terminal data. Brief Description of the Drawings

[0014] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0015] Figure 1 It is a flowchart of the method for constructing a digital twin model of a mobile terminal according to the present invention; Figure 2 It is a system block diagram of the system for constructing a digital twin model of a mobile terminal in this embodiment; Figure 3 It is a schematic diagram of non-relational multi-source heterogeneous mobile terminal data in this embodiment. Detailed Embodiments

[0016] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0017] This embodiment provides a method for constructing a digital twin model of a mobile terminal, as Figure 1 shown, which includes the following steps: S1: Collect and obtain non-relational multi-source heterogeneous mobile terminal data; And the multi-source heterogeneous mobile terminal data includes mobile terminal application data, hardware sensor data, and user private data; Specifically, as the functions of smartphones continue to increase and users' dependence on mobile phones continues to deepen, it is used as a personal daily data collection terminal. At the same time, to ensure the security of user data while using it, all collected user data needs to be locally stored on the mobile terminal. By combining existing technologies, it is possible to collect and completely cover non-relational multi-source heterogeneous data of personal life and work fragments. In this embodiment, personal data, that is, mobile terminal data, is divided into the following three categories, as Figure 3 shown: First, application data, which is the data generated by users using the application services provided by various application vendors; second, sensor data, which is the data collected based on the sensors and communication modules inside and outside the intelligent terminal; third, user private data, which is the data such as users' thoughts, records, and behaviors generated at any time; Among them, application data: Since the application data used daily is distributed among major application vendors and does not provide a data export function, and the technical frameworks used by each system application are different, it is extremely difficult to collect, obtain, and integrate application-related data through traditional data network crawlers; This embodiment collects application data on mobile intelligent terminal devices in a what-you-see-is-what-you-get manner, and the collection methods of application data include an active collection method of application data based on Accessibility Service and a passive collection method of application data; Active collection is the collection of personal data of third-party applications, and the active collection method of application data specifically includes: S001: Enable the permissions of the Accessibility Service of the Android system: Include enabling the permission to perform gesture operations (can Perform Gestures) for setting simulated click operations on the mobile terminal; Enable the permission to retrieve window content (can Retrieve Window Content) for setting to obtain interface data; S002: Open a floating window or an active window for collecting mobile terminal application data; S003: Call the background server to download an application data collection plugin to collect and obtain the structured data of the mobile terminal's application data based on the simulated click operation according to the position of the floating window; Specifically, it includes the following steps: S0031: Start the third-party APP according to the set third-party APP package name; S0032: Obtain the tool (get Root In ActiveWindow) through the interface activity window node of the accessibility service, obtain the root node of the accessibility tree in the current active window, so as to obtain the similarity between the current active window node tree and the expected set interface node tree of the final data interface of the third-party APP, and then determine whether the current active window interface already belongs to the expected set interface of the final data interface of the third-party APP; The expected set interface of the final data interface of the third-party APP is the interface used to provide identification for the final data interface; The method of obtaining the similarity between the current active window node tree and the expected set interface node tree of the final data interface of the third-party APP is a well-known technical means in the art and will not be elaborated here; S0033: If the current active window interface belongs to the expected set interface of the final data interface of the third-party APP of the mobile terminal, the next-level function page of the current active window is the final data interface; If the current active window interface does not belong to the expected set interface of the final data interface of the third-party APP of the mobile terminal, enter the next-level function page of the current active window, and repeat S132 until the current active window interface belongs to the expected set interface of the final data interface of the third-party APP of the mobile terminal, and obtain the final data interface; S0034: If the data of the final data interface is displayed in a list or loaded in real time through the network, execute S0035; If the final data interface is displayed in the form of pictures, execute S0036; S0035: Scroll down the final data interface to turn pages until there is no new data or the collection requirement is reached; According to the method of getting Root In Active Window of the interface activity window node of the accessibility service, traverse all node elements of the final data interface to obtain the large string of all text data of the final data interface; S0036: Store the pictures locally or on a remote server, and splice the picture paths to form the large string of all picture paths of the final data interface; S0037: Based on the large string and regular expressions, obtain the structured data of the final data interface of the third-party APP of the mobile terminal.

[0018] In this embodiment, the passive collection of application data refers to some information that users see when using the APP, including data received during activities such as chatting, browsing news, online shopping, and swiping through Moments. By collecting the data that users view in real time, we can effectively understand the latest knowledge information received by users and identify the recent status of users, etc. The traditional method of collecting third-party application data is through OCR recognition after taking screenshots. However, due to mobile phone performance issues, it is not possible to effectively and real-time collect all the information that users see. Therefore, this application proposes a method for passive collection of application data based on Accessibility Service, and the method for passive collection of application data specifically includes: S100: Enable the permission of the Accessibility Service of the Android system: The mobile terminal is specifically a mobile phone with the Android system; It includes enabling the permission to perform gesture operations (can Perform Gestures) for setting simulated click operations on the mobile terminal; Enable the permission to retrieve window content (can Retrieve Window Content) for setting to obtain interface data; S101: Listen for changes in the mobile terminal page in real time by calling the on Accessibility Event tool of the Accessibility Service. When the page changes, collect the structured data displayed on the mobile terminal page in real time by calling the get RootIn Active Window tool for the interface activity window node; otherwise, do not perform the collection action; Method for listening for changes in the mobile terminal page in real time by calling the on Accessibility Event tool of the Accessibility Service: S1011: Obtain the click moment when the mobile terminal receives an interface click event and use it as the first click moment; Obtain the second click moment when the mobile terminal receives an interface click event, and the second click moment is the click moment corresponding to the interface click event after the interface click event corresponding to the first click moment; S1012: Obtain the click time interval between the second click moment and the first click moment; Judge the size between the click time interval and the preset time interval threshold; If the click time interval is greater than the preset time interval threshold, confirm that the mobile terminal page has changed; if the click time interval is less than or equal to the preset time interval threshold, confirm that the mobile terminal page has not changed; Hardware sensor data: Hardware data can accurately detect and identify the physical environment where the user is located. Through NFC tags, it can precisely transmit the user's real-time scenario to the set digital twin space. Through WIFI and Bluetooth connections, data can determine the physical space location of the user. Through data processing algorithms, it can identify the user-defined scenarios for data such as GPS, gyroscope, gravity sensor, and light sensor. Moreover, by combining external intelligent hardware sensors, it can enrich the state collection of the physical world where the user is located, enabling more intelligent assistance that conforms to the user's physical state in the digital twin space.

[0019] User private data: It is provided that users can record all private data in their lives in the form of text, voice, photos, and videos. Through the convenient and fast information recording method, the collection of this type of data can help users digitize the previous unconscious information, and can use artificial intelligence large model technology to visualize this type of data in the digital twin space, providing solid basic data for the personal digital twin. In a specific embodiment, S1 also includes a method for data compression of multi-source heterogeneous mobile terminal data: Encoding the source symbols in the multi-source heterogeneous mobile terminal data based on Huffman coding to obtain source coding symbols; Obtaining the occurrence probability of each source coding symbol, and performing data compression on the multi-source heterogeneous mobile terminal data according to the source coding symbols based on the probability statistical model of the source; During the actual data collection process, there may occasionally be a problem that the local data processing speed cannot keep up with the collection speed. This solution performs vector data retrieval on the data of the user's concerned dimensions, that is, by classifying the collected mobile terminal data according to the user's concern level, that is, the application scenarios set by the user and recognized and matched by the backend, and adopting a data compression method to transmit it to the large model for processing at a fixed time according to the priority corresponding to the data type. In this embodiment, it is considered that due to the characteristic that the same source symbol content frequently appears in the user's daily data, the data source of the user's daily data is a core concept in the fields of data science, information technology, and communication, referring to the original source of data or the entity / system that provides data. It can be any medium, device, platform, or environment that generates, stores, or transmits data, and the quality, reliability, and type of the data source directly affect the application value and analysis results of the data; Therefore, this embodiment adopts Huffman coding and defines the probability according to the frequency of the source symbol appearing in the data, so as to achieve the purpose of data compression based on the probability statistical model of the source; And the expression of the probability statistical model based on the source is , In the formula: Represents the expected length of data compression; Represents the average code length; Represents the coding length; E represents expectation; Represents the description of summing over all possible source coding symbols ; Ξ represents the set of source coding symbols; Represents the source coding symbol The probability of occurrence; Represents the source symbol The corresponding coding length; where source coding symbols with high probability are encoded with short codes, and source symbols with low probability are encoded with long codes; S2: Construct a three-dimensional memory mapping relationship model; And based on the three-dimensional memory mapping relationship model, store the multi-source heterogeneous mobile terminal data, obtain a three-dimensional memory mapping relationship table for storing non-relational data, and redefine the field names of the multi-source heterogeneous mobile terminal data as the field indexes corresponding to the non-relational mobile terminal data fields through the three-dimensional memory mapping table; Perform vectorization processing on the multi-source heterogeneous mobile terminal data obtained through the field index, obtain vectorized data for storage and obtain a mobile terminal database; In this embodiment, the application of the standard digital twin technology mainly focuses on using a high-precision three-dimensional model to accurately replicate and model the target object in the physical world on a one-to-one basis. It involves extensive data collection work, that is, densely collecting data from each key position point of the physical target entity to ensure that a highly consistent and complete entity mapping can be constructed in the virtual digital space. However, the complexity and diversity of the physical world make the model data show multi-dimensional characteristics. Especially for entities with complex structures, diverse functions, and data-driven, it is far from enough to only consider their basic forms for modeling. The needs of users for digital twin applications also often have highly personalized characteristics and need to flexibly customize the models in the virtual space according to their actual needs. In view of this, this embodiment is based on a comprehensive data collection technology, and through constructing a three-dimensional memory mapping relationship model, processes and integrates various generalization data in the daily use of users to obtain comprehensive information about user preferences, behavior habits, and potential needs; Since the data sources collected in this embodiment are very extensive, including dimensions such as active and passive, real-time and historical, full volume and incremental, and the data types are diverse, covering structured, unstructured, and semi-structured data, and these data specifically include various forms such as text, voice, pictures, and videos. In order to better construct a digital twin space model based on these data, effective strategies need to be adopted to preprocess these data. The current conventional method is to store the data in the database after standard structuring. However, due to the extremely complex data collection data sources, some data integrity and scalability will be lost. User-related private data cannot participate in the model construction due to dimensional uncertainty. At the same time, since the original data needs to be stored to provide subsequent modeling, display, analysis, use, circulation, etc., it is necessary to format and store multi-source heterogeneous data. The existing solution is that non-relational data can be used to dynamically increase and decrease fields and data tables. However, under the condition of relatively weak computing performance of mobile terminals, the choice of non-relational databases is relatively limited, and it may be difficult to understand the meaning of the field content when using these databases. Therefore, by constructing a three-dimensional memory mapping relationship model, data processing and formatted storage of multi-source heterogeneous mobile terminal data can be achieved; In a specific embodiment, S2 specifically includes the following steps: S21: Construct a three-dimensional memory mapping relationship model, which includes a data table setting module, a field set acquisition module, a data type specification module, and a three-dimensional memory mapping relationship module; The data table setting module is used to set the data data table for storing multi-source heterogeneous mobile terminal data according to the set SQLite database; The field set acquisition module is used to add data fields to the multi-source heterogeneous mobile terminal data stored in the data data table to obtain a data field set F; And F = {data_0, data_1,..., data_n}; where data_n represents the nth added data field; The data type specification module is used to set a type label set for each data field type in the data field set; The three-dimensional memory mapping relationship module is used to construct a three-dimensional memory mapping relationship formula according to the data field set and the type label set, and obtain a three-dimensional memory mapping table through the three-dimensional memory mapping relationship formula; And the three-dimensional memory mapping relation formula is M: F×T type×N→V; where M represents the abbreviated form of M(f,t,n); Ttype represents the set of type tags; N represents the unique identifier of the data record; V represents the data value containing all the multi-source heterogeneous mobile terminal data to be stored; f represents the data field and f∈F; t represents the data type and t∈T type; n represents the value corresponding to the record identifier and n∈N; in this embodiment, due to the flexibility of F and T type, the model can store data from different sources and these data can be heterogeneous, and can support standard SQL query data; through the M mapping in this embodiment, the relationship between each data type field can be easily queried, updated and maintained, and preprocessing is performed when using standard SQL data query. By redefining the field name as the field index corresponding to the field through the three-dimensional memory mapping model, the potential value of the data can be more fully utilized in the subsequent data usage process; S22: Redefine the field name of the multi-source heterogeneous mobile terminal data as the field index corresponding to the non-relational mobile terminal data field through the obtained three-dimensional memory mapping table; S23: Based on the pre-trained continuous bag-of-words model CBOW, perform vectorization processing on the multi-source heterogeneous mobile terminal data obtained through the field index, obtain the vectorized data for storage and obtain the mobile terminal database; Specifically, in this embodiment, by adopting word2vec combined with a custom dictionary and word vector method, the text data collected by the mobile terminal and the slicing of other unstructured or semi-structured data using the TIKA tool are converted into vectors and mapped and stored with the original data; when using word2vec, considering the problem of insufficient computing performance of the current mobile terminal, the Embedding-improved Word2vec technology is introduced to vectorize the multi-source heterogeneous mobile terminal data using the CBOW (Continuous Bag-of-WordsModel) continuous bag-of-words model pre-trained with relevant data, and directly select a specific row from the weight matrix and pass it to the next layer, avoiding the long matrix multiplication operation; and the role of the Embedding layer is to extract the row (vector) corresponding to the word ID from the weight parameters, thereby reducing the amount of calculation. At the same time, combined with the common domain dictionary in the user's daily life and work, the usability and practicality of the data in subsequent modeling are greatly improved; In this embodiment, a relational database SQLite that is more suitable for mobile terminal embedding is used to store non-relational data. By creating a data data table and adding data fields data_0 to data_63, it can support up to 64 data types of fields at most. And by adding a data_type field to specify the type of the data, a three-dimensional memory mapping relationship model is constructed to index the original fields, so as to achieve better storage of multi-source heterogeneous data similar to the characteristics of columnar databases, better maintain the relationship between various data type fields, and be more suitable for OLAP (Online Analytical Processing) scenarios. This embodiment not only enhances the extensibility of data types and fields, but also through a preset data system or users can customize the data format of any data type to achieve formatted storage; S3: Provide a data retrieval service for the mobile terminal database to implement the data retrieval operation of vectorized data in the mobile terminal database; In a specific embodiment, the data retrieval service provided for the mobile terminal database includes a data retrieval service based on semantic retrieval and a data retrieval service based on SQL retrieval; Specifically, the traditional keyword matching algorithm only considers the lexical similarity and ignores the relationship between sentence structure and semantics. The semantic retrieval algorithm can better understand the user's query intention and provide more accurate search results. And in this embodiment, in order to accelerate the vector retrieval speed, a local retrieval is performed on the relevant processed mobile terminal data through an embedded vector search engine J Vector accelerated by graph algorithms and SIMD (Single Instruction, Multiple Data); The retrieval method of the data retrieval service based on semantic retrieval: Based on the mobile terminal database, several data vector search navigation layer nodes for data retrieval are constructed through the HNSW graph algorithm; And the edge weights of the data vector search navigation layer nodes are obtained based on the heuristic search method; According to the edge weights and the data vector search navigation layer nodes, a search navigation graph structure for data retrieval is obtained; Based on the embedded vector search engine J Vector of single instruction multiple data streams SIMD, data indexing of the mobile terminal database is realized according to the search navigation graph structure; Specifically, in this embodiment, through the large language model and Text2SQL combined with the local optimized data storage method technology, it can automatically analyze the user's problem and convert it into a corresponding SQL query statement, simplifying the database query process, improving the accuracy and efficiency of the query, reducing the threshold of database operations, enabling more users to conveniently obtain and utilize structured data, and being able to complete complex database query work without professional SQL knowledge; Retrieval method of the data retrieval service based on SQL retrieval: Semantically parse the data in the mobile terminal database through the large language model LLM to obtain a structured query intent representation; Based on the pre-trained Text2SQL model, obtain an SQL statement according to the structured query intent representation; Through the local data storage method technology, based on the SQL statement, obtain an SQL query statement according to the storage format of the local database and the field index characteristics of the vectorized data, so as to realize the data indexing of the mobile terminal database according to the SQL query statement; S4: Construct a virtual entity mapping space based on the vectorized data obtained from the data retrieval operation; Define an application agent applied to the virtual entity mapping space, and obtain a mobile terminal digital twin model for realizing real-time perception processing and execution of RPA in the user environment according to the virtual entity mapping space; In the process of constructing a virtual model of a physical entity applied to a digital twin space in this embodiment, traditional methods usually use Unity 3D / Unreal Engine for pre-modeling, and realize a comprehensive synchronous mapping with the virtual entity by collecting relevant data of the physical entity in real time. This method enables users to intuitively master the overall state of the physical entity through the virtual entity, and then perform effective control, prediction and other operations. Its advantage lies in providing a comprehensive understanding of the physical entity. However, when modeling a digital twin space for entities related to users' daily work and life on a mobile terminal, it does not have good scalability. Users can only select the system to construct a digital twin for the pre-modeled physical entity, and can only display the preset basic information of the system. It is impossible to apply digital twin technology to all aspects of users' lives to meet users' personalized needs; Specifically, it includes the following steps: S41: Customize the entity model and entity events of any mobile terminal; Customize and construct a Prompt text for obtaining a structured semantic description framework according to the entity model and entity events of the mobile terminal through natural language technology; S42: Invoke the AIGC model of text-to-image / video-to-video, and obtain the representation of the virtual digital entity model according to the Prompt text; S43: Convert the Prompt text into a query vector to be queried, and based on the data retrieval service provided by the mobile terminal database, retrieve and obtain multi-source heterogeneous mobile terminal data related to the representation of the virtual digital entity model through the query vector to be queried; Combined with a pre-trained large language model (LLM), a summary report of entity events is obtained based on multi-source heterogeneous mobile terminal data retrieved and acquired. The summary report of the entity events is the virtual entity mapping space where the user is located. S44: Define application agents applied to the virtual entity mapping space; for example, information security agents and health management agents, and based on the summary report of entity events retrieved and acquired from the virtual entity mapping space, determine the execution action instructions for a certain scenario in the current user environment. S45: Perform actions on the mobile terminal according to the execution action instructions to implement the function of real-time perception processing and execution of RPA in the user environment, thereby realizing the construction of the mobile terminal digital twin model; and the execution action instructions at least include operation instructions for the user's mobile terminal interface and access instructions for the remote server interface. In this embodiment, through the Accessibility Service, operations such as clicking, swiping, and inputting on the interface can be performed to implement the RPA function, providing action recording for more convenient operations for users. Enabling a transparent floating window can record the click coordinate path of the user, record the user's actions, and associating the actions with the application scenario can achieve linkage. For example, when the digital twin space recognizes that the scenario is going home and connecting to the home's automatic WIFI, and senses that the current environmental temperature is not suitable, the action is to automatically find the Mijia application and then turn on the air conditioner. Through large model scenario recognition, it solves the traditional rule-based scenario automation problem and breaks the linkage limitations between various applications. This embodiment adopts AIGC technology, allowing users to customize the entity model or thing dimension of the mobile terminal. By customizing the construction of Prompts through the user's natural language, virtual digital entities are constructed using text-to-image / video. At the same time, semantic retrieval of the collected and vectorized relevant data is performed through the custom natural language of the Prompt to obtain all relevant original data of the described entity, and combined with the large language model LLM to implement the complete function of RAG (Retrieval-augmented Generation) for the local data of the mobile terminal. The retrieved and matched data is processed by the large model to obtain a summary report of the digital entity. In addition, users can redefine the Prompt to enable the large model to further process and return the dimension information that the user cares about, and provide placeholders for real-time dimension information to be embedded in the large model's answer results to maintain high-speed information synchronization. At the same time, users can further view and export the detailed structured data related to the entity model through the provided data retrieval function. This embodiment also includes a mobile terminal digital twin model construction system, as Figure 2As shown in the figure, it includes a data acquisition module, a data compression module, a three-dimensional memory mapping relationship model construction module, a data vectorization module, a data retrieval service module, an application agent setting module, and a mobile terminal digital twin model construction module; The data acquisition module is used to collect and obtain non-relational multi-source heterogeneous mobile terminal data; And the multi-source heterogeneous mobile terminal data includes mobile terminal application data, hardware sensor data, and user data; The data compression module is used to compress the multi-source heterogeneous mobile terminal data to obtain compressed data; The three-dimensional memory mapping relationship model construction module is used to construct a three-dimensional memory mapping relationship model; And based on the three-dimensional memory mapping relationship model, the multi-source heterogeneous mobile terminal data is stored to obtain a three-dimensional memory mapping relationship table for storing non-relational data, and the field names of the multi-source heterogeneous mobile terminal data are redefined as the field indexes corresponding to the non-relational mobile terminal data fields through the three-dimensional memory mapping table; The data vectorization module is used to perform vectorization processing on the multi-source heterogeneous mobile terminal data obtained through the field index, obtain vectorized data for storage, and obtain a mobile terminal database; The data retrieval service module is used to provide a data retrieval service for the mobile terminal database to implement the data retrieval operation of the vectorized data in the mobile terminal database; And the data retrieval service includes a data retrieval service based on semantic retrieval and a data retrieval service based on SQL retrieval; The application agent setting module is used to set application agents applied to the virtual entity mapping space, and the application agents at least include an information security agent and a health management agent; The mobile terminal digital twin model construction module is used to construct a virtual entity mapping space according to the vectorized data obtained by the data retrieval operation, and based on the application agent, obtain a mobile terminal digital twin model for realizing real-time perception processing and execution of RPA in the user environment according to the virtual entity mapping space.

[0020] Model Application of the Digital Twin Model of the Mobile Terminal Constructed in this Embodiment: Through the various data collected in the daily life of individual users, users can customize the complete digital twin virtual mapping of various entities in life, view and control the detailed status of the entity, and at the same time provide users with the application of the entire twin space data. Based on the pre-training on large-scale text data, the large language model has mastered rich language knowledge and patterns. These models can understand complex language structures, generate coherent and logical texts, and then enable the large language model to parse the queries or prompts input by users, understand the intentions and needs behind them, and thus make accurate analyses and decisions. For example, when a user needs movie recommendations, a semantic analysis model constructed through the user's daily relevant corpus retrieves vectors of relevant data such as the user's preferences, status, and movies according to the user's question, submits them to the large language model for processing and response, and at the same time provides various data AI AGENTS (intelligent agents) to conduct more in-depth mining of user data. For example, the information security intelligent agent can detect all the real-time collected user data and give a security report and repair suggestions, discover whether the user's current behavior is abnormal, and find out whether the user is being scammed through the real-time collected interface text conversation content. The health management intelligent agent comprehensively analyzes the data of the user's daily diet, exercise, physiology, sleep, etc. collected and gives accurate analyses based on real and complete data.

[0021] Beneficial Effects of the Method in this Embodiment: In this embodiment, a three-dimensional memory mapping relationship model is constructed to store multi-source heterogeneous mobile terminal data, and a three-dimensional memory mapping relationship table for storing non-relational data is obtained, effectively realizing the collection and integration of fragmented information of non-relational multi-source heterogeneous mobile terminal data, and greatly improving the comprehensiveness of obtaining the user's mobile terminal numbers; through the three-dimensional memory mapping table, the field names of multi-source heterogeneous mobile terminal data are redefined as the field indexes corresponding to non-relational mobile terminal data fields, and the multi-source heterogeneous mobile terminal data obtained through the field indexes is vectorized to obtain vectorized data for storage and obtain a mobile terminal database, effectively realizing the structured storage of multi-source heterogeneous mobile terminal data, laying a foundation for providing users with a complete data export service; by defining application intelligent agents applied to the virtual entity mapping space and obtaining the construction of a digital twin model of the mobile terminal for realizing real-time perception processing and execution of RPA in the virtual entity mapping space according to the constructed virtual entity mapping space, it provides technical support for users in aspects such as managing their own data security, maintaining data rights and interests, processing and using data, and understanding data status, and greatly improves the management and application of mobile terminal data.

[0022] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for constructing a digital twin model of a mobile terminal, characterized in that: The following steps are involved: S1: Collect and obtain non-relational multi-source heterogeneous mobile terminal data; And the multi-source heterogeneous mobile terminal data includes mobile terminal application data, hardware sensor data and user private data; S2: Construct a three-dimensional memory mapping relationship model; Based on the three-dimensional memory mapping relationship model, the multi-source heterogeneous mobile terminal data is stored, a three-dimensional memory mapping relationship table for storing non-relational data is obtained, and the field names of the multi-source heterogeneous mobile terminal data are redefined as field indexes corresponding to the non-relational mobile terminal data fields through the three-dimensional memory mapping table; Vectorize the multi-source heterogeneous mobile terminal data obtained through field indexing, obtain the vectorized data to store and obtain the mobile terminal database; S3: Provide data retrieval services for the mobile terminal database to implement data retrieval operations for vectorized data in the mobile terminal database; S4: constructing a virtual entity mapping space based on the vectorized data obtained by the data retrieval operation; Define the application agent applied to the virtual entity mapping space, and obtain the digital twin model of the mobile terminal for realizing real-time perception and processing of the user environment and executing RPA based on the virtual entity mapping space.

2. A method for constructing a digital twin model of a mobile terminal according to claim 1, characterized in that: The S2 specifically includes the following steps: S21: constructing a three-dimensional memory mapping relationship model, which includes a data table setting module, a field set acquisition module, a data type specifying module and a three-dimensional memory mapping relationship module; The data table setting module is used to set a data table for storing multi-source heterogeneous mobile terminal data according to the set SQLite database; The field set acquisition module is used to add data fields to obtain a data field set F for the multi-source heterogeneous mobile terminal data stored in the data table; And F={data_0,data_1,…,data_n}; where data_n represents the nth data field added; The data type specifying module is used to set a type tag set for each data field type in the data field set; The three-dimensional memory mapping relationship module is used to construct a three-dimensional memory mapping relationship according to a data field set and a type label set, and obtain a three-dimensional memory mapping table through the three-dimensional memory mapping relationship; And the three-dimensional memory mapping relationship is M:F×T type×N→V; where M represents the abbreviation of M(f, t, n); Ttype represents the type tag set; N represents the unique identifier of the data record; V represents the data value containing all multi-source heterogeneous mobile terminal data to be stored; f represents the data field and f∈F; t represents the data type and t∈T type; n represents the value corresponding to the record identifier and n∈N; S22: redefine the field names of the multi-source heterogeneous mobile terminal data as field indexes corresponding to the non-relational mobile terminal data fields through the obtained three-dimensional memory mapping table; S23: Based on the pre-trained continuous bag of words model CBOW, vectorize the multi-source heterogeneous mobile terminal data obtained through field indexing, obtain vectorized data to store and obtain the mobile terminal database.

3. A method for constructing a digital twin model of a mobile terminal according to claim 1, characterized in that: The data retrieval service in S3 includes data retrieval service based on semantic retrieval and data retrieval service based on SQL retrieval; The retrieval method of the data retrieval service based on semantic retrieval: Based on the mobile terminal database, several data vector index navigation layer nodes for data retrieval are constructed through the HNSW graph algorithm; And based on the heuristic search method, the edge weights of the nodes in the data vector index navigation layer are obtained; According to the edge weight and data vector index navigation layer nodes, obtain the index navigation graph structure for data retrieval; J Vector, an embedded vector search engine based on SIMD, implements data indexing of mobile terminal databases according to the index navigation graph structure; The retrieval method of the data retrieval service based on SQL retrieval: The data in the mobile terminal database is semantically parsed through the large language model (LLM) to obtain structured query intent representation; Based on the pre-trained Text2SQL model, SQL statements are obtained according to the structured query intent representation; Through local data storage technology, based on SQL statements, SQL query statements are obtained according to the storage format of the local database and the field index characteristics of vectorized data, so as to implement data indexing of the mobile terminal database according to the SQL query statements.

4. A method for constructing a digital twin model of a mobile terminal according to claim 3, characterized in that: The S4 specifically comprises the following steps: S41: Customize entity models and entity events of any mobile terminal; Based on the entity model and entity events of the mobile terminal, a prompt text for obtaining a structured semantic description framework is customized through natural language technology. S42: calling the AIGC model of the Wensheng picture / Wensheng video, and obtaining the virtual digital entity model representation according to the Prompt text; S43: converting the prompt text into a vector to be queried, and based on a data retrieval service provided by a mobile terminal database, retrieving and acquiring multi-source heterogeneous mobile terminal data related to the representation of the virtual digital entity model through the vector to be queried; In combination with the pre-trained large language model LLM, an overview report of entity events is obtained based on the retrieved and acquired multi-source heterogeneous mobile terminal data. The overview report of entity events is the virtual entity mapping space where the user is located; S44: defining an application agent applied to the virtual entity mapping space, and determining an execution action instruction for a certain scene in the current user environment based on an overview report of entity events retrieved and obtained by the virtual entity mapping space; S45: Execute the action of the mobile terminal according to the execution action instruction to realize the function of real-time perception and processing of the user environment and execution of the RPA, thereby realizing the construction of the digital twin model of the mobile terminal; And the execution action instruction at least includes an operation instruction for the user mobile terminal interface and an access instruction for the remote server terminal interface.

5. A method for constructing a digital twin model of a mobile terminal according to claim 4, characterized in that: The method for collecting and acquiring application data of the mobile terminal in S1 includes an active application data collection method based on accessibility services and a passive application data collection method; The active application data collection method: S001: Enable the accessibility service permissions of the Android system: Including enabling the permission setting of allowing gesture operations to be performed for simulating click operations on the mobile terminal; Enable the permission setting of allowing traversal of window contents to obtain interface data; S002: Open a floating window for collecting mobile terminal application data; S003: calling the background server to download the application data collection plug-in, so as to collect and obtain the structured data of the application data of the mobile terminal according to the location of the floating window based on the simulated click operation; The passive collection method of application data: S100: Enable the accessibility service permissions of the Android system: Including enabling the permission setting of allowing gesture operations to be performed for simulating click operations on the mobile terminal; Enable the permission setting of allowing traversal of window contents to obtain interface data; S101: monitoring the changes of the mobile terminal page in real time by calling the accessibility service monitoring event tool, and when the page changes, collecting the structured data displayed in real time on the mobile terminal page by calling the interface active window node acquisition tool; otherwise, no collection action is performed; Method to call the accessibility service event monitoring tool to monitor the changes of mobile terminal pages in real time: S1011: Acquire the click time when the mobile terminal receives the interface click event, and use it as the first click time; Acquire a second click time when the mobile terminal receives the interface click event, where the second click time is the click time corresponding to the interface click event after the interface click event corresponding to the first click time; S1012: Obtaining a click time interval between the second click moment and the first click moment; Determine the size of the click time interval and the preset time interval threshold; If the click time interval is greater than the preset time interval threshold, it is confirmed that the mobile terminal page has changed; if the click time interval is less than or equal to the preset time interval threshold, it is confirmed that the mobile terminal page has not changed.

6. A method for constructing a digital twin model of a mobile terminal according to claim 1, characterized in that: S1 also includes a method for compressing multi-source heterogeneous mobile terminal data: Encode the source symbols in the multi-source heterogeneous mobile terminal data based on Huffman coding to obtain source coding symbols; Obtain the occurrence probability of each source coding symbol, and perform data compression on multi-source heterogeneous mobile terminal data according to the source coding symbols through a source-based probability statistical model; And the expression of the probability statistical model based on the information source is , Where: Indicates the expected length of data compression; represents the average code length; Indicates the encoding length; E indicates expectation; Represents all possible source coding symbols A description of the sum is performed; Ξ represents a set of source coding symbols; Indicates source coding symbol Probability of occurrence; Indicates source coding symbol The corresponding encoding length.

7. A system based on the method for constructing a digital twin model of a mobile terminal according to any one of claims 1 to 6, characterized in that: It includes data acquisition module, data compression module, three-dimensional memory mapping relationship model construction module, data vectorization module, data retrieval service module, application intelligent agent setting module, and mobile terminal digital twin model construction module; The data acquisition module is used to collect and acquire non-relational multi-source heterogeneous mobile terminal data; And the multi-source heterogeneous mobile terminal data includes mobile terminal application data, hardware sensor data and user data; The data compression module is used to compress multi-source heterogeneous mobile terminal data to obtain compressed data; The three-dimensional memory mapping relationship model construction module is used to construct a three-dimensional memory mapping relationship model; Based on the three-dimensional memory mapping relationship model, the multi-source heterogeneous mobile terminal data is stored, a three-dimensional memory mapping relationship table for storing non-relational data is obtained, and the field names of the multi-source heterogeneous mobile terminal data are redefined as field indexes corresponding to the non-relational mobile terminal data fields through the three-dimensional memory mapping table; The data vectorization module is used to perform vectorization processing on multi-source heterogeneous mobile terminal data obtained through field indexing, obtain vectorized data to store and obtain a mobile terminal database; The data retrieval service module is used to provide data retrieval services to the mobile terminal database to implement data retrieval operations on vectorized data in the mobile terminal database; And the data retrieval service includes a data retrieval service based on semantic retrieval and a data retrieval service based on SQL retrieval; The application agent setting module is used to set the application agent applied to the virtual entity mapping space, and the application agent at least includes an information security agent and a health management agent; The mobile terminal digital twin model construction module is used to construct a virtual entity mapping space based on the vectorized data obtained by the data retrieval operation, and based on the application agent, obtain a mobile terminal digital twin model for realizing real-time perception processing of the user environment and executing RPA.

Images