Intelligent equipment access system and access method based on AI large model

Abstract

The invention relates to an intelligent equipment access system and access method based on an AI large model, and relates to the field of artificial intelligence and Internet of Things. According to the technical scheme, a dynamically adaptive universal object model access framework is constructed, functional attributes, control instructions and data formats of intelligent devices of different brands and protocols are mapped into a standardized logic model, meanwhile, a multi-mode AI large model engine is combined, high-precision voice recognition, voiceprint feature detection and multi-level semantic intention analysis capabilities are integrated, and the intelligent devices of different brands and protocols can be obtained. And an intelligent interaction closed loop of end-cloud collaboration is formed. Semantic analysis adopts a mixed strategy, a natural language instruction is converted through function mapping, a local knowledge base is utilized to match user scene requirements, and external data is queried by means of cloud networking to optimize instruction logic. According to the scheme, the multi-device access and operation and maintenance cost is remarkably reduced, the semantic understanding accuracy and scene adaptability are improved, and the method is widely applied to the fields of smart home, smart medical treatment and industrial Internet of Things.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence and Internet of Things technology, and in particular to an intelligent device access system and access method based on an AI big model. Background Art

[0002] In today's era of booming smart devices, different brands and manufacturers have launched various smart products in an attempt to build an intelligent living and working environment. However, the protocol heterogeneity problem between smart devices in the existing technology is like a huge gap, which seriously hinders the collaboration between devices. Because different brands and manufacturers use different communication protocols and data formats, it is difficult to achieve seamless connection and data sharing between different devices. For example, in a smart home scenario, a user's home may have a smart lighting system of brand A, smart home appliances of brand B, and smart security equipment of brand C. However, due to the heterogeneity of protocols, these devices cannot communicate with each other and work together. Users cannot centrally manage all the devices in the house through a unified control terminal, and can only operate the exclusive applications of each brand separately, which greatly limits the scalability and flexibility of the equipment, making it impossible for smart devices to give full play to their due integration advantages and difficult to build an efficient and convenient smart home ecosystem. Not only that, for users, facing the unique operation methods and interaction logic of devices of different brands, it takes a lot of time to learn and adapt, which undoubtedly increases the learning cost of users. When equipment fails or needs to be upgraded, the operation and maintenance methods and technical requirements of equipment from different brands are different, which further increases the difficulty of operation and maintenance and consumes a lot of user time and energy.

[0003] At the same time, the existing interaction methods of smart devices also have obvious shortcomings. At present, smart devices mainly rely on fixed control instruction formats, such as command line mode. This interaction method is extremely simple and not intuitive enough. For ordinary users, the command line mode requires memorizing complex instruction syntax and parameter settings, and the operation threshold is high, making it difficult to get started easily. Taking a smart speaker as an example, if a user wants to let the speaker play a specific song through voice commands, in the command line mode, it may be necessary to strictly follow the specific instruction format, such as "play song, singer name is [singer name], song name is [song name]", once the expression is slightly deviated, the speaker will not be able to accurately recognize the instruction. Moreover, the existing technology has serious deficiencies in semantic understanding ability. When users issue complex or ambiguous instructions, smart devices often find it difficult to accurately understand the user's intentions and perform corresponding operations. For example, if a user says "I feel a little cold, turn up the temperature in the living room a little bit", the smart device may not be able to accurately judge whether the user means adjusting the air conditioning temperature or turning on the electric heater, etc., resulting in a significant reduction in user experience. These pain points have seriously restricted the popularization and development of smart devices, and there is an urgent need for an innovative technical solution to break these bottlenecks, and the present invention is based on this. Summary of the invention

[0004] The purpose of the present invention is to provide an intelligent device access system and access method based on an AI big model to solve the problems of high heterogeneity of intelligent device protocols, single interaction mode and insufficient semantic understanding ability existing in the above-mentioned prior art.

[0005] To achieve the above object, the technical solution adopted by the present invention is:

[0006] In a first aspect, the present invention provides an intelligent device access system based on an AI big model, the system comprising:

[0007] The device access layer is used to access smart devices, collect data and monitor the status of the smart devices, and dynamically optimize the protocol parsing path through reinforcement learning;

[0008] The universal object model layer is used to standardize the data of smart devices of different brands and protocols in the device access layer, build a standardized universal object model, and realize the collaborative update of the edge-cloud protocol model in combination with the federated learning framework;

[0009] The intelligent interaction processing layer is used to parse user instructions, fuse voice, text and environmental sensor data using a cross-modal attention mechanism, and generate control instructions;

[0010] The data security and assurance layer is used to integrate the abnormal instruction self-repair engine based on the knowledge graph and dynamically verify the legitimacy of the instructions; and

[0011] The application and presentation layer is used to provide operation interfaces and management tools for end users and operation and maintenance personnel;

[0012] Among them, the device access layer provides raw data and status information to the universal object model layer, and the universal object model layer standardizes the raw data and status information for use by the intelligent interaction processing layer; the intelligent interaction processing layer ensures the accuracy and security of the data through the data security and assurance layer, and the data analysis results output by the data security and assurance layer are used to optimize the understanding and responsiveness of the intelligent interaction processing layer; the application and display layer displays the interaction results of the intelligent interaction processing layer in a terminal user-friendly manner, and feeds back the terminal user's instructions and operations to the operation and maintenance management background for recording.

[0013] In a second aspect, the present invention provides an intelligent device access method based on an AI large model, the method being applied to the intelligent device access system based on the AI ​​large model as described above, the method comprising:

[0014] S1. Receive voice or text instructions sent by the terminal user, call the universal object model layer, map it to the standardized universal object model according to the data format and protocol of the smart device, match the smart device type through the universal object model, and determine the corresponding properties, events and methods;

[0015] S2, in response to receiving a voice command sent by a terminal user, enabling a voice recognition module to convert the voice command into a text command;

[0016] S3. Based on the text instruction, calling the semantic parsing and intention understanding module, and analyzing the real needs of the terminal user in a multimodal manner;

[0017] S4. Based on the actual needs of the terminal user, a control instruction of the smart device is generated and sent to the target smart device, and the operation result is fed back to the terminal user through the user interaction interface.

[0018] In a third aspect, the present invention provides a computer device, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the processor can load and execute at least one instruction, at least one program, a code set or an instruction set to implement the smart device access method based on the AI ​​big model provided above.

[0019] The beneficial effects brought about by the technical solution provided by the present invention include at least:

[0020] This technical solution builds a dynamically adaptive universal object model access framework, maps the functional attributes, control instructions and data formats of smart devices of different brands and protocols into standardized logical models, and realizes "plug and play" of devices and unified cross-platform management and control. At the same time, combined with a multimodal AI large model engine, it integrates high-precision speech recognition, voiceprint feature detection and multi-level semantic intent analysis capabilities to form an intelligent interactive closed loop of end-cloud collaboration. Semantic analysis adopts a hybrid strategy, converts natural language instructions through function mapping, uses the local knowledge base to match user scenario requirements, and uses cloud networking to query external data to optimize instruction logic, such as "raising the temperature" combined with real-time room temperature to determine the target value. In addition, the system introduces voiceprint feature extraction and security verification modules to ensure the legitimacy and privacy of instructions. This solution significantly reduces the cost of multi-device access and operation and maintenance, improves the accuracy of semantic understanding and scenario adaptability, and is widely applicable to smart home, smart medical and industrial Internet of Things fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0022] Figure 1 A structural block diagram of an intelligent device access system based on an AI big model provided by an exemplary embodiment of the present invention is shown.

[0023] Figure 2 A flowchart of a smart device access method based on an AI big model provided by an exemplary embodiment of the present invention is shown.

[0024] Figure 3 A logical block diagram of an intelligent device access method based on an AI big model provided by an exemplary embodiment of the present invention is shown.

[0025] Figure 4 A structural schematic diagram of a computer device for executing an intelligent device access method based on an AI big model provided by an exemplary embodiment of the present invention is shown. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0028] Figure 1The structure block diagram of an intelligent device access system based on an AI big model provided by an exemplary embodiment of the present invention is shown. The intelligent device access system based on the AI ​​big model includes: a device access layer, a universal object model layer, an intelligent interaction processing layer, a data security and protection layer, and an application and display layer. The device access layer is the bottom layer of the system. The device access layer is used to access various intelligent devices (such as home appliances, sensors, wearable devices, etc.), and collect data and monitor the status of intelligent devices, and dynamically optimize the protocol parsing path through reinforcement learning; the universal object model layer is used to standardize the data of intelligent devices of different brands and protocols in the device access layer, and build a standardized universal object model (supporting "plug and play" and cross-platform management), and combine the federated learning framework to realize the collaborative update of the edge-cloud protocol model; the intelligent interaction processing layer is the core of the system, combined with multimodal AI technology, it is used to parse user-indicated The cross-modal attention mechanism is used to fuse voice, text and environmental sensor data, and generate control instructions; the data security and protection layer is used to integrate the abnormal instruction self-repair engine based on the knowledge graph, and dynamically verify the legitimacy of the instructions; the top application and display layer is used to provide an operation interface and management tools for end users and operation and maintenance personnel; among them, the device access layer provides raw data and status information to the general object model layer, and the general object model layer standardizes the raw data and status information for use by the intelligent interaction processing layer; the intelligent interaction processing layer ensures the accuracy and security of the data through the data security and protection layer, and the data analysis results output by the data security and protection layer are used to optimize the understanding and responsiveness of the intelligent interaction processing layer; the application and display layer displays the interaction results of the intelligent interaction processing layer in a terminal user-friendly manner, and feeds back the terminal user's instructions and operations to the operation and maintenance management background for recording.

[0029] Specifically, the device access layer includes: a physical layer protocol adapter module, and a device status acquisition module. The physical layer protocol adapter module is used to access different smart devices and perform data analysis; the device status acquisition module is used to collect and feedback the operating status and properties of smart devices in real time.

[0030] In some embodiments, the physical layer protocol adaptation module of the device access layer includes at least heterogeneous protocol parsing modules such as Zigbee / Bluetooth / Wi-Fi / StarFlash, and compatibility encapsulation modules of brand-specific SDKs; among them, heterogeneous protocol parsing modules such as Zigbee / Bluetooth / Wi-Fi / StarFlash can cope with various mainstream and emerging communication protocols on the current market; for example, the Zigbee protocol is often used for low-power, self-organizing smart home device connections, and the protocol parsing module can accurately interpret the data transmitted; Bluetooth is widely used in short-range device interactions, such as the connection between smart bracelets and mobile phones, and the module can quickly adapt and parse the instructions transmitted by Bluetooth; Wi-Fi, as a common network connection method, supports the access of many smart home appliances to the network, and the module's analysis of its data ensures smooth networking of the device; and StarFlash, as an emerging wireless short-range communication technology, the module can also provide strong support for the access of related devices. The compatibility packaging module of the brand-specific SDK solves the access difficulties caused by the unique designs of devices of different brands. In order to ensure product characteristics and security, many brands will develop exclusive software development kits (SDKs). This module uses compatibility packaging to enable these brand devices to be smoothly integrated into the system and achieve unified management.

[0031] In some embodiments, the device status collection module of the device access layer includes at least a real-time collection module for device function attributes, status, data and other information, and a device event receiving module. The real-time collection module for device function attributes, status, data and other information continuously monitors various operating parameters of the device, such as the temperature setting, operating mode, real-time energy consumption and other data of the smart air conditioner, and provides the system with real-time status information of the device; the device event receiving module is always on standby to capture various events generated during the operation of the device, such as the opening and closing events of the smart door lock and the alarm events of the smoke sensor, so that the system can respond and process them in a timely manner.

[0032] Furthermore, the universal object model layer includes: a data layer protocol adaptation and encapsulation module, a dynamic adaptation module, and an access gateway module. The data layer protocol adaptation and encapsulation module is used to map the functional attributes, control instructions and data formats of smart devices of various brands to the universal object model; the dynamic adaptation module is used to maintain the online and offline status of the device and update the universal object model in real time; the access gateway module serves as a data transfer and logical bridge to complete data routing and coordination.

[0033] In an embodiment of the present application, the data layer protocol adaptation encapsulation module optimizes the protocol matching efficiency through a reinforcement learning algorithm, specifically including: constructing a dynamic reward function to generate a real-time reward value based on the historical protocol parsing success rate, latency, and energy consumption; using a deep Q network (DQN) to select the optimal protocol parsing path, and dynamically adjusting the gradient descent weight update strategy; when a new smart device is connected, calling the offline pre-trained protocol compatibility prediction model to reduce the initial matching error rate to below 3%.

[0034] More specifically, the data layer protocol adaptation and encapsulation module optimizes the protocol matching efficiency through a reinforcement learning algorithm, including:

[0035] Dynamic reward function construction: Generate real-time reward value based on historical protocol parsing success rate, latency and energy consumption indicators, and define the reward function as Among them, S t is the analysis success rate, D t is the delay (ms), E t is the energy consumption (mW), α, β, γ are weight coefficients;

[0036] Deep Q Network (DQN) path selection: For newly connected smart devices, the feature vectors of their initial communication data packets (such as data frame length and response delay distribution) are extracted and input into a lightweight neural network model (Topology ID: T-ADP-RL). The optimal protocol parsing path is predicted through the DQN algorithm, and the mapping weights between smart devices and general object models are dynamically adjusted to reduce the initial matching error rate to less than 3%;

[0037] Protocol compatibility prediction model: Using an offline pre-trained LSTM network (Model ID: M-PCP-001), the compatibility probability is predicted based on the device model, protocol version, and historical matching records. If the probability is lower than the threshold (e.g. <90%), the manual review process is triggered; and

[0038] Version compatibility includes: a dynamic version rule base, which defines the universal object model version compatibility rule base and supports dynamic loading of conversion logic according to the device firmware version; a seamless mapping mechanism, which automatically loads the historical version universal object model converter (VerConv module) if it detects that the device firmware version is lower than the system requirement (such as V1.2), and compensates for missing parameters through an interpolation algorithm to ensure that old device instructions (such as V1.2's "brightness adjustment" instructions) are seamlessly mapped to the current universal object model standard (V2.1's "lightness parameter"); rollback protection, when a version upgrade causes a compatibility conflict, it automatically rolls back to the most recent stable version and records fault logs for cloud analysis.

[0039] In one example, the data layer protocol adaptation and encapsulation module is deployed as an independent server to centrally handle the protocol adaptation and encapsulation tasks of all smart devices. This method is suitable for scenarios with a large number of devices and centralized management, and can achieve efficient protocol conversion and data routing through powerful computing resources. In another example, the data layer protocol adaptation and encapsulation module is deployed as an embedded edge gateway hardware device, which is deployed in a network close to smart devices; when the data layer protocol adaptation and encapsulation module is deployed as an embedded edge gateway hardware device, the federated learning framework is used to optimize the protocol parsing model, including: each edge gateway trains a lightweight protocol parsing model locally, and encrypts and uploads the model parameters to the cloud; the cloud aggregates multi-node parameters to generate a global optimization model, which is sent to the edge gateway; and the local data is protected by differential privacy technology to ensure user privacy security. The edge gateway can complete protocol adaptation and data processing locally, reduce dependence on the central server, and achieve load balancing. This method is particularly suitable for scenarios with widely distributed devices or sensitive network delays, and can significantly improve the response speed and reliability of the system. By flexibly selecting the form of the data layer protocol adaptation and encapsulation module, the needs of different scenarios can be better met, and efficient access and management of smart devices can be achieved.

[0040] In the embodiment of the present application, the general object model adopts JSON or XML structured data format, including: basic device information, attributes, events, and methods. The basic device information at least includes the smart device ID, smart device name, smart device type (such as air conditioner, light bulb, sensor, etc.), and manufacturer information; attributes are used to describe the readable and writable status and parameters of the smart device (such as temperature, wind speed, switch status, etc.); events indicate state changes or abnormal notifications that may be triggered by the smart device during operation (such as temperature exceeding the set value, device offline, etc.); methods indicate the operations or functions supported by the smart device (such as "increase the temperature", "turn on", "turn off", etc.).

[0041] In the embodiments of the present application, the universal object model supports dynamic expansion, allowing new device types to be added or the functions of existing devices to be updated while the system is running. The dynamic adaptation module updates the universal object model in real time according to the actual status of the device, ensuring that the system can adapt to real-time changes in the device. The universal object model defines a unified interface standard to ensure that smart devices from different manufacturers and with different protocols can be mapped to the universal object model through the data layer protocol adaptation encapsulation module, which includes data transmission format, command request / response format, etc., to ensure "plug and play" and cross-platform management of smart devices. The universal object model is closely integrated with the intelligent interaction processing layer to support the interaction requirements of multimodal AI technology. The methods in the universal object model can be mapped to smart device control instructions, which are converted into specific control functions through semantic parsing based on the natural language instructions of the end user.

[0042] In some embodiments, the data layer protocol adapter encapsulation module of the universal object model layer includes at least a protocol parsing module, a data conversion module, and an error handling and logging module. The protocol parsing module further performs an in-depth analysis of the protocol data transmitted from the device access layer to accurately extract useful information; the data conversion module uniformly converts data of different formats into a format that meets the requirements of the universal object model to ensure smooth transmission and use of data within the system; the error handling and logging module can quickly take measures to correct errors once they are encountered during data processing, and record error information in detail to facilitate subsequent investigation and optimization.

[0043] In some embodiments, the dynamic adaptation module of the general object model layer includes at least a device registration module, a status detection module, and a device offline processing module. The device registration module is responsible for registering information for newly connected devices in the system and establishing device files; the status detection module monitors the online status of the device regularly or in real time to ensure that the device is always within the controllable range; the device offline processing module performs corresponding operations in a timely manner when the device is offline, such as saving the status of the device before offline, cleaning up temporary data, etc., to ensure that the device can be seamlessly connected when it goes online again.

[0044] In some embodiments, the access gateway module of the general object model layer includes at least a data routing module, a cache management module, and a secure transmission module. The data routing module rationally plans the data transmission path according to the device requirements and system rules to ensure that the data is efficiently delivered to the destination; the cache management module caches the data to improve the data access speed and cope with network fluctuations; the secure transmission module uses encryption and other technical means to ensure the security of data during transmission and prevent data leakage and tampering.

[0045] Specifically, the intelligent interaction processing layer includes: a speech recognition module, a voiceprint feature extraction and security verification module, and a semantic analysis and intent understanding module. The speech recognition module is used to convert the terminal user's voice commands into text information; the voiceprint feature extraction and security verification module is used to verify the terminal user's identity and ensure the legitimacy of the commands; the semantic analysis and intent understanding module is used to parse the commands and convert them into control commands for smart devices.

[0046] In an embodiment of the present application, the semantic parsing and intention understanding module adopts a cross-modal attention network, which specifically includes: a speech feature extraction sub-module, which analyzes acoustic features through Mel spectrum graphs; a text semantic encoding sub-module, which uses the BERT model to generate semantic vectors; an environmental sensor data fusion sub-module, which collects and encodes light and temperature data in real time; a cross-modal attention layer, which calculates the association weights of speech, text, and environmental sensor data, and generates environmental perception parsing results; when the user instruction contains fuzzy parameters, the instruction logic "dim the lights" is dynamically corrected in combination with the environmental sensor data to calculate the brightness target based on the current light value).

[0047] Specifically, the speech feature extraction submodule extracts the acoustic features of the speech command through Mel spectrum analysis (frequency band 0-8kHz, frame length 25ms) and generates a 128-dimensional feature vector; the text semantic encoding submodule uses a trimmed version of the BERT model (parameter quantity <1M) to encode the text command and output a 256-dimensional semantic vector; the environmental sensor data fusion submodule collects environmental sensor data (such as light value, temperature value) in real time, and encodes it into a 32-dimensional feature vector after normalization; the cross-modal attention layer calculates the association weight formula of speech, text, and sensor data as follows: Where Q i is the query vector (text semantics), is the key vector (voice / sensor), d is the dimension; dynamic command correction, when the command parameters are ambiguous (such as "dim the lights"), the target brightness is calculated according to the sensor data (current light value) and the control command is output.

[0048] In some embodiments, the speech recognition module of the intelligent interaction processing layer includes at least a feature extraction submodule, an acoustic model submodule, a language model submodule, and a decoding submodule. The feature extraction submodule first analyzes the input speech signal, extracts key features such as frequency and amplitude, and converts the original speech into a feature vector for subsequent processing, laying the foundation for subsequent steps. The acoustic model submodule is trained based on a large amount of speech data, and can associate the extracted feature vector with the corresponding speech content, and identify the basic units such as phonemes and syllables in the speech through complex algorithms. The language model submodule considers the grammatical rules, semantic logic, and common expressions of the language, and uses this knowledge to optimize the results of the acoustic model output, eliminate ambiguity, and improve the accuracy of speech recognition. The decoding submodule combines the results of the acoustic model and the language model, finds the most suitable result from many possible speech recognition candidates through a search algorithm, and accurately converts the speech into text information.

[0049] In some embodiments, the voiceprint feature extraction and security verification module of the intelligent interaction processing layer includes at least a voiceprint extraction engine, a voiceprint database, and a voiceprint comparison engine. The voiceprint extraction engine performs in-depth analysis on the user's voice and extracts unique voiceprint features. These features are like each person's voice fingerprint and have a high degree of individual specificity. The voiceprint database stores the voiceprint information of authorized users in the system. When a new voice command is input, the voiceprint comparison engine quickly compares the extracted voiceprint features with the voiceprints in the database to verify the user's identity. Only commands that pass the verification will be allowed to enter the next step of processing, thereby effectively preventing illegal users from manipulating smart devices and ensuring the security of the system and user privacy.

[0050] In some embodiments, the semantic parsing and intention understanding module of the intelligent interaction processing layer includes at least a function mapping module, a local knowledge base matching module, a cloud data fusion module, and a control decision module. The function mapping module can accurately map the natural language instructions input by the user to the corresponding device control function based on the general object model of the device, such as mapping "increasing the air conditioner temperature" to the temperature increase control function of the air conditioner. The local knowledge base matching module will query the locally stored device function information, user historical operation records, scene rules and other knowledge, and combine the current instructions to determine whether the instructions are in line with the current usage scenario, providing localized support for understanding user intentions. When the local knowledge base information is insufficient, the cloud data fusion module connects to the cloud through the network to obtain external data such as real-time weather and user location, further enriching the understanding dimension of the instructions and optimizing the instruction logic. The control decision module combines the analysis results of the previous modules to finally determine the user's true intention, and generates corresponding smart device control decisions to guide the system to accurately control the smart devices.

[0051] More specifically, the data security and assurance layer includes: local knowledge base management module, cloud data integration and calculation module, data security module, and log and monitoring module. The local knowledge base management module is used to manage smart device information, scenario rules and mapping relationships; the cloud data integration and calculation module is used to integrate external data and analyze it to provide support for the intelligent interaction processing layer; the data security module is used to ensure the security of identity authentication, access control and data encryption transmission; the log and monitoring module is used to monitor the system operation status and terminal user instructions in real time to provide an optimization basis. Among them, the log and monitoring module integrates an abnormal instruction self-repair engine. When a control instruction fails to execute, it automatically calls the historical successful instruction log to generate an alternative instruction, and combines smart device status verification and cloud deep analysis to achieve multi-level fault tolerance.

[0052] In some embodiments, the local knowledge base management module of the data security and protection layer includes at least a device information management module, a scenario rule base module, and a mapping relationship management module. The device information management module manages the detailed parameters, functional characteristics, maintenance records and other information of various smart devices in a fine manner to ensure that the device-related data can be accurately called at any time during the operation of the system; the scenario rule base module stores the rules set for different application scenarios, such as the device linkage rules for different time periods in the smart home scenario, or the device control rules for specific production processes in the industrial Internet of Things, providing a basis for the system's intelligent decision-making; the mapping relationship management module focuses on maintaining the correspondence between device data, user instructions and system operations, so that the entire system can accurately transmit and process information during the data interaction process.

[0053] In some embodiments, the cloud data integration and computing module of the data security and protection layer includes at least an external data interface module, a data analysis module, and a real-time computing and prediction module. The external data interface module actively establishes connections with various external data sources, widely collects multi-dimensional data such as market dynamics, industry trends, weather information, etc., and provides the system with richer information resources; the data analysis module uses advanced data mining and analysis algorithms to conduct in-depth analysis of massive data, explore the potential value behind the data, and provide strong data support for system optimization; the real-time computing and prediction module performs rapid calculations based on real-time data, and uses machine learning models to predict equipment operating status, user behavior, etc., to discover potential problems in advance, and help the system achieve intelligent preventive maintenance and management.

[0054] In some embodiments, the data security module of the data security and protection layer includes at least an identity authentication module, an access control module, and an encryption transmission module. The identity authentication module strictly confirms the user's identity through a variety of authentication methods, such as password authentication, fingerprint recognition, and facial recognition, to ensure that only legitimate users can access system resources; the access control module carefully divides the access levels to different system functions and data according to the user's role and authority to prevent the risk of data leakage caused by unauthorized operations; the encryption transmission module uses advanced encryption algorithms to encrypt data at every stage of the transmission process to effectively prevent data from being stolen or tampered with.

[0055] In some embodiments, the log and monitoring module of the data security and protection layer includes at least a system operation monitoring module, a user instruction log module, and a device status log module. The system operation monitoring module monitors various performance indicators of the system in real time, such as server load, network bandwidth usage, data processing speed, etc. Once an abnormality is found, an alarm is immediately issued so that the operation and maintenance personnel can handle it in time; the user instruction log module records in detail every operation instruction of the user in the system, including instruction content, execution time, execution results and other information, providing detailed data for subsequent user behavior analysis and problem troubleshooting; the device status log module focuses on recording the changes in the operation status of smart devices, such as the power on, shutdown, fault alarm and other events of the device, helping the operation and maintenance personnel to fully understand the operation history of the device and provide an important basis for the maintenance and management of the device.

[0056] In detail, the application and display layer includes: user interaction interface, operation and maintenance management background, and open interface services. The user interaction interface is used to support multiple interaction methods, which include at least mobile applications, web consoles and voice assistants; the operation and maintenance management background is used to provide operation and maintenance personnel with smart device management and status monitoring tools; the open interface service is used to provide APIs for third-party developers to support data interoperability and function expansion.

[0057] In some embodiments, the user interaction interface of the application and display layer includes at least a mobile application module, a web console module, and a voice assistant integration module. The mobile application module of the user interaction interface fully takes into account the needs of modern users for mobile office and life, allowing users to interact efficiently with the system anytime, anywhere. The web console module provides users with an operating platform with more comprehensive functions and a clearer layout. Users can access the web console through a browser on the computer to perform in-depth settings and complex operations on the system. It is suitable for scenarios that require centralized management of a large number of devices or fine parameter adjustments. The voice assistant integration module brings a new interactive experience to users. Users can easily achieve device control, information query and other operations through voice commands. The voice assistant can quickly and accurately identify user intentions and convey commands to the system.

[0058] In some embodiments, the operation and maintenance management background of the application and display layer includes at least a device monitoring module, a task scheduling module, and a data statistics and analysis module. The device monitoring module presents the operating status of various intelligent devices in real time, including the online / offline status of the device, working parameters, fault warning and other information, so that the operation and maintenance personnel can grasp the dynamics of the device at the first time, and promptly discover and deal with potential problems. The task scheduling module is responsible for the reasonable arrangement of various tasks in the system, and intelligently schedules the order and time of task execution according to factors such as the priority of the device and resource occupancy, to ensure efficient and stable operation of the system and avoid task conflicts and waste of resources. The data statistics and analysis module collects, organizes and analyzes the massive data generated during the operation of the system, generates various intuitive data reports and trend charts, and provides decision-making basis for operation and maintenance personnel, helping them optimize system performance and formulate reasonable maintenance plans.

[0059] In some embodiments, the open interface service of the application and presentation layer includes at least a RESTful API module, a WebSocket real-time communication module, an SDK and a document module. The RESTful API module follows the REST architectural style, and uses a concise and standardized interface design to facilitate third-party developers to call various functions and data of the system, and realize interconnection and data sharing between different systems. The WebSocket real-time communication module is committed to providing a real-time, two-way communication channel, which enables third-party applications to interact with the system in real time, and meet application scenarios with high real-time requirements such as real-time monitoring and message push. The SDK and document modules provide developers with a complete set of development toolkits and detailed interface documents to lower the development threshold, help developers get started quickly, and more efficiently carry out secondary development based on the system, enriching the application scenarios and functional ecology of the system.

[0060] Figure 2 A flowchart of a smart device access method based on an AI big model provided by an exemplary embodiment of the present invention is shown. Figure 3 A logic block diagram of an AI large model-based smart device access method provided by an exemplary embodiment of the present invention is shown. The AI ​​large model-based smart device access method is applied to the AI ​​large model-based smart device access system as described above. The method includes the following steps:

[0061] Step S1, receiving the voice or text command sent by the terminal user, calling the universal object model layer, mapping it to the standardized universal object model according to the data format and protocol of the smart device, matching the smart device type through the universal object model, and determining the corresponding properties, events and methods.

[0062] For example, the attributes of an air conditioner may include temperature and wind speed; an event may be a temperature change notification; and methods may include "increase the temperature" or "turn off the device." This standardized processing enables devices of different brands and protocols to be managed and controlled in a unified manner, thus achieving the goal of "plug and play" and cross-platform control.

[0063] In an embodiment of the present application, receiving voice or text instructions sent by the terminal user is the starting point of the entire interactive process. Whether it is a text instruction entered by the terminal user through a mobile phone APP or a voice instruction spoken through a smart speaker, it will be acquired by the system. The universal object model layer is then called, and the universal object model layer "translates" and maps the user instructions to a standardized universal object model based on the data format and protocol of these smart devices. Through this standardized processing, the system can easily match the type of smart device. For example, for smart door lock devices, it can be determined through matching that it has attributes such as unlocking and locking, events such as abnormal opening of door locks, and methods such as "remote unlocking". This unified processing method allows different devices to be centrally managed and controlled by the system, communicate and manage through unified rules, and ultimately achieve the purpose of "plug and play" and cross-platform control, which is convenient for terminal users to operate freely between different devices and platforms without worrying about compatibility issues.

[0064] Step S2: In response to receiving a voice instruction sent by a terminal user, a voice recognition module is enabled to convert the voice instruction into a text instruction.

[0065] In the embodiment of the present application, step S2 also includes: for voice commands, using the voiceprint feature extraction and security verification module to compare the voice features of the terminal user with the pre-stored voiceprint information to determine the legitimacy of the command and the identity of the terminal user. This ensures that only authorized users can control the device. This security mechanism effectively prevents unauthorized operations and protects user privacy and system security.

[0066] In the embodiment of the present application, when the system receives a voice command, the voice recognition module is enabled to convert the voice command into a text command, which is an important step in achieving human-computer interaction. The voice recognition module converts the words spoken by the terminal user into text, and uses the voiceprint feature extraction and security verification module to compare the voice features of the terminal user with the pre-stored voiceprint information. The system saves the voiceprint information of the authorized user in advance. When a new voice command comes, the module will extract the sound features and compare them with the pre-stored voiceprint. Only when the voiceprint matches, the command is deemed legal and the user's identity is correct. Otherwise, the command will be rejected, thereby effectively preventing others from arbitrarily manipulating the device, protecting the user's privacy and the security of the entire system, and making users feel more at ease when using smart devices.

[0067] Step S3: Based on the text instructions, the semantic parsing and intention understanding module is called, and the real needs of the terminal users are analyzed in a multimodal manner.

[0068] In the embodiment of the present application, step S3 specifically includes: performing function parsing based on the universal object model, mapping the text instruction to a predefined function in the universal object model (for example, the instruction "increase the temperature" will be mapped to the "increase the temperature" method of the air-conditioning equipment); querying the local knowledge base, and judging whether the instruction conforms to the current usage scenario based on the smart device function information, scenario rules and user historical behavior (for example, the system will judge whether "increase the temperature" conforms to the current usage scenario based on the user's usage habits); in response to the lack or insufficiency of knowledge base content, querying external data through the cloud interface to optimize the instruction logic (for example, when the user says "increase the temperature", the system will dynamically calculate the target temperature value based on the current room temperature, external temperature and the user's historical preferences), and the knowledge base parsing and online query are independent and complementary to each other (knowledge base parsing is mainly used for localized fast matching and scenario-based demand judgment, while online query is used to supplement the missing or insufficient content in the knowledge base; for example, in the smart elderly care scenario, the knowledge base can enter relevant knowledge of the elderly care site, and when the system needs more specific health advice, the latest health information or health data can be obtained through online query). Knowledge base parsing and online query work together, one is responsible for fast local matching and scene judgment, and the other is responsible for supplementing rich external information, jointly improving the accuracy and comprehensiveness of the system's understanding of user intent.

[0069] Step S4: Based on the actual needs of the terminal user, a control instruction of the smart device is generated and sent to the target smart device, and the operation result is fed back to the terminal user through the user interaction interface.

[0070] In the embodiment of the present application, step S4 also includes: for asynchronous operations, notifying the terminal user of status updates through the same interactive method after the operation is completed.

[0071] In an embodiment of the present application, the operation results are fed back to the terminal user through the user interaction interface, which is the key to maintaining a good user experience. For example, when the terminal user issues an instruction to "adjust the TV volume", the system generates an instruction and sends it to the TV, and then informs the user on the mobile phone APP or smart speaker that "the TV volume has been adjusted to 50%". For asynchronous operations, such as the washing process of a smart washing machine and the baking process of a smart oven, these operations take a certain amount of time to complete. After the operation is completed, the system notifies the terminal user of the status update through the same interactive method, such as sending a message to inform the user that "the washing machine has completed the washing program" or "the oven baking has ended". In this way, the user does not need to pay attention to the operating status of the device at all times, which improves the convenience of use.

[0072] Through the above steps, the smart device and the AI ​​big model have achieved a complete interactive closed loop from receiving and parsing user commands to controlling and feedback of the device. Knowledge base parsing and online query play independent and complementary roles in semantic understanding, further improving the intelligence level and scenario adaptability of the system. This not only improves the user experience, but also ensures the security and efficiency of the system, and can be widely used in smart homes, smart medical care, industrial Internet of Things and other fields.

[0073] Figure 4 A schematic diagram of the structure of a computer device for executing an intelligent device access method based on an AI large model provided by an exemplary embodiment of the present invention is shown, and the computer device includes:

[0074] The processor 401 includes one or more processing cores. The processor 401 executes various functional applications and data processing by running software programs and modules.

[0075] The receiver 402 and the transmitter 403 can be implemented as a communication component, and the communication component can be a communication chip. Optionally, the communication component can be implemented to include a signal transmission function. That is, the transmitter 403 can be used to transmit a control signal to the image acquisition device and the scanning device, and the receiver 402 can be used to receive the corresponding feedback instruction.

[0076] The memory 404 is connected to the processor 401 via a bus 405 .

[0077] The memory 404 may be used to store at least one instruction, and the processor 401 may be used to execute the at least one instruction to implement each step in the above method embodiment.

[0078] Unless otherwise stated, all technical and scientific terms used in this disclosure have the same meaning as those generally understood by those skilled in the art of the technical field of this specification. The terms used in this specification are only for the purpose of describing specific embodiments and are not intended to limit the scope of this specification. The term "and / or" used in this specification includes any and all combinations of one or more related listed items. The singular forms of "a", "above", and "the" used in this disclosure and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0079] It can be understood that the processor of the present disclosure can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method implementation method can be completed by the hardware integrated logic circuit or software instructions in the processor. The above processor can be a general processor, a digital signal processor (DigitalSignalProcessor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the present disclosure can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the present disclosure can be directly embodied as a hardware decoding processor to perform, or the hardware and software modules in the decoding processor can be combined to perform. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0080] It is understood that the memory in the present disclosure may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memory. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (programmable ROM, PROM), an erasable programmable read-only memory (erasable PROM, EPROM), an electrically erasable programmable read-only memory (EEPROM) or a flash memory. The volatile memory may be a random access memory (RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0081] The above is only a specific implementation of this specification, but the protection scope of the present invention is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in this specification, which should be included in the protection scope of this specification. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. An intelligent device access system based on AI big model, characterized in that: The system comprises: The device access layer is used to access smart devices, collect data and monitor the status of the smart devices, and dynamically optimize the protocol parsing path through reinforcement learning; The universal object model layer is used to standardize the data of smart devices of different brands and protocols in the device access layer, build a standardized universal object model, and realize the collaborative update of the edge-cloud protocol model in combination with the federated learning framework; The intelligent interaction processing layer is used to parse user instructions, fuse voice, text and environmental sensor data using a cross-modal attention mechanism, and generate control instructions; The data security and assurance layer is used to integrate the abnormal instruction self-repair engine based on the knowledge graph and dynamically verify the legitimacy of the instructions; and The application and presentation layer is used to provide operation interfaces and management tools for end users and operation and maintenance personnel; Among them, the device access layer provides raw data and status information to the universal object model layer, and the universal object model layer standardizes the raw data and status information for use by the intelligent interaction processing layer; the intelligent interaction processing layer ensures the accuracy and security of the data through the data security and assurance layer, and the data analysis results output by the data security and assurance layer are used to optimize the understanding and responsiveness of the intelligent interaction processing layer; the application and display layer displays the interaction results of the intelligent interaction processing layer in a terminal user-friendly manner, and feeds back the terminal user's instructions and operations to the operation and maintenance management background for recording.

2. The intelligent device access system based on AI big model according to claim 1 is characterized in that: The device access layer includes: Physical layer protocol adapter module, which is used to access different smart devices and analyze data; and The device status acquisition module is used to collect and feedback the operating status and properties of smart devices in real time.

3. The intelligent device access system based on AI big model according to claim 1 is characterized in that: The general object model layer includes: Data layer protocol adaptation and encapsulation module, which is used to map the functional attributes, control instructions and data formats of smart devices of various brands to the universal object model; A dynamic adaptation module, which is used to maintain the online and offline status of the device and update the universal object model in real time; and The access gateway module serves as a data transfer and logical bridge to complete data routing and coordination.

4. The AI ​​big model-based smart device access system according to claim 3 is characterized in that: The data layer protocol adaptation encapsulation module optimizes the protocol matching efficiency through a reinforcement learning algorithm, specifically including: Build a dynamic reward function to generate real-time reward values ​​based on historical protocol parsing success rate, latency, and energy consumption; Use a deep Q network to select the optimal protocol parsing path and dynamically adjust the gradient descent weight update strategy; When a new smart device is connected, the offline pre-trained protocol compatibility prediction model is called to reduce the initial matching error rate to below 3%.

5. The AI ​​big model-based smart device access system according to claim 3 is characterized in that: The data layer protocol adaptation and encapsulation module is deployed as an independent server to centrally process the protocol adaptation and encapsulation tasks of all smart devices; or The data layer protocol adaptation and encapsulation module is deployed as an embedded edge gateway hardware device, deployed in a network close to the smart device; Among them, when the data layer protocol adaptation encapsulation module is deployed as an embedded edge gateway hardware device, a federated learning framework is used to implement protocol parsing model optimization, specifically including: Each edge gateway trains a lightweight protocol parsing model locally and uploads the model parameters to the cloud in encrypted form; The cloud aggregates multi-node parameters to generate a global optimization model and sends it to the edge gateway; Protect local data through differential privacy technology to ensure user privacy.

6. The AI ​​big model-based smart device access system according to claim 3 is characterized in that: The general object model adopts JSON or XML structured data format, including: Basic device information, which at least includes smart device ID, smart device name, smart device type, and manufacturer information; Attributes, which are used to describe the readable and writable status and parameters of smart devices; Events, which indicate state changes or abnormal notifications that may be triggered during the operation of smart devices; and Methods, which indicate the operations or functions supported by the smart device.

7. The intelligent device access system based on AI big model according to claim 1 is characterized in that: The intelligent interaction processing layer includes: A speech recognition module, which is used to convert the terminal user's voice commands into text messages; Voiceprint feature extraction and security verification module, which is used to verify the identity of the end user and ensure the legitimacy of the instructions; and The semantic parsing and intention understanding module is used to parse instructions and convert them into control commands for smart devices.

8. The AI ​​big model-based smart device access system according to claim 7 is characterized in that: The semantic parsing and intention understanding module adopts a cross-modal attention network, which specifically includes: The speech feature extraction submodule analyzes acoustic features through Mel spectrogram; The text semantic encoding submodule uses the BERT model to generate semantic vectors; Environmental sensor data fusion submodule collects and encodes light and temperature data in real time; The cross-modal attention layer calculates the association weights of speech, text, and environmental sensor data to generate environmentally aware parsing results; When user instructions contain fuzzy parameters, the instruction logic is dynamically corrected in combination with environmental sensor data.

9. The AI ​​big model-based smart device access system according to claim 1, characterized in that: The data security and protection layer includes: Local knowledge base management module, which is used to manage smart device information, scene rules and mapping relationships; A cloud data integration and computing module, which is used to integrate and analyze external data and provide support for the intelligent interaction processing layer; Data security module, which is used to ensure the security of identity authentication, access control and data encryption transmission; and The log and monitoring module is used to monitor the system operation status and terminal user instructions in real time, providing an optimization basis; Among them, the log and monitoring module integrates an abnormal instruction self-repair engine. When a control instruction execution failure is detected, it automatically calls the historical successful instruction log to generate a replacement instruction, and combines the intelligent device status verification and cloud-based deep analysis to achieve multi-level fault tolerance.

10. The AI ​​big model-based smart device access system according to claim 1, characterized in that: The application and presentation layer includes: A user interaction interface, which is used to support multiple interaction modes, including at least a mobile application, a web console, and a voice assistant; The operation and maintenance management backend is used to provide intelligent device management and status monitoring tools for operation and maintenance personnel; and Open interface service, which is used to provide API for third-party developers to support data interoperability and function expansion.

11. A smart device access method based on AI big model, characterized in that: The method is applied to the smart device access system based on the AI ​​big model according to any one of claims 1 to 10, and the method comprises: S1. Receive voice or text instructions sent by the terminal user, call the universal object model layer, map it to the standardized universal object model according to the data format and protocol of the smart device, match the smart device type through the universal object model, and determine the corresponding properties, events and methods; S2, in response to receiving a voice command sent by a terminal user, enabling a voice recognition module to convert the voice command into a text command; S3. Based on the text instruction, calling the semantic parsing and intention understanding module, and analyzing the real needs of the terminal user in a multimodal manner; S4. Based on the actual needs of the terminal user, a control instruction of the smart device is generated and sent to the target smart device, and the operation result is fed back to the terminal user through the user interaction interface.

12. The method for accessing smart devices based on AI big model according to claim 11, characterized in that: The step S2 further comprises: For voice commands, the voiceprint feature extraction and security verification module is used to compare the terminal user's voice features with the pre-stored voiceprint information to determine the legitimacy of the command and the identity of the terminal user.

13. The method for accessing smart devices based on AI big model according to claim 11, characterized in that: The step S3 specifically includes: Perform function parsing based on a universal object model, and map the text instruction to a predefined function in the universal object model; Query the local knowledge base and determine whether the instruction is suitable for the current usage scenario based on the smart device function information, scenario rules and user historical behavior; In response to missing or insufficient knowledge base content, external data is queried through the cloud interface to optimize instruction logic, and knowledge base analysis and online query are independent and complementary.

14. The method for accessing smart devices based on AI big model according to claim 11, characterized in that: The step S4 further comprises: For asynchronous operations, the end user is notified of status updates through the same interactive method after the operation is completed.

Images