Response method, deployment method, system and device based on large language model

Abstract

The invention provides a response method, a deployment method, a system and a device based on a large language model, and relates to the technical field of artificial intelligence, in particular to the fields of Internet of Things, edge computing and natural language processing. According to the specific implementation scheme, in response to received input information of a target object, according to the mode of the input information, a preset processing mode corresponding to the mode is adopted to process the input information, and processed data is obtained; determining reference data from the data acquisition equipment, wherein the reference data represents the equipment state of the data acquisition equipment or the environment information of the environment where the data acquisition equipment is located; processing the processed data and the reference data by using a large language model to obtain response information for the input information; wherein the large language model is locally deployed on the edge side equipment based on model resources.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to the fields of Internet of Things, edge computing, and natural language processing. More specifically, the present disclosure provides a response method, an application resource deployment method, an information processing system, a response device, an application resource deployment device, an electronic device, a storage medium, and a computer program product. Background Art

[0002] With the development of artificial intelligence, relevant technologies of artificial intelligence can be used to process problem and requirement information input by users, so as to answer questions raised by users or provide solutions to requirements. In practical applications, users expect to obtain more intelligent responses in real time, and expect various input methods and received response forms. Summary of the Invention

[0003] The present disclosure provides a response method, an application resource deployment method, a response device, an application resource deployment device, an electronic device, a storage medium, a computer program product, and an information processing system.

[0004] According to one aspect of the present disclosure, there is provided a response method, including: in response to receiving input information of a target object, processing the input information according to a predetermined processing mode corresponding to the modality of the input information to obtain processed data; determining reference data from a data acquisition device, where the reference data represents the device state of the data acquisition device or the environmental information of the environment where the device acquisition device is located; and using a large language model to process the processed data and the reference data to obtain a response information for the input information; wherein the large language model is deployed locally on an edge device based on model resources.

[0005] According to another aspect of the present disclosure, there is provided an application resource deployment method, including: obtaining device information of a plurality of edge devices in an edge device; determining a matching relationship between the plurality of edge devices and a plurality of application resources according to the device information of the plurality of edge devices and requirement information, where the plurality of application resources includes model resources; and based on the matching relationship, sending resource information of the matching application resources to the plurality of edge devices, so that the edge device deploys the matching application resources locally and executes the above response method based on the matching application resources.

[0006] According to another aspect of the present disclosure, a response device is provided, including: a processing module, a reference data determination module, and a response module. The processing module is configured to, in response to receiving input information of a target object, process the input information according to a predetermined processing mode corresponding to the modality of the input information to obtain processed data. The reference data determination module is configured to determine reference data from a data acquisition device, where the reference data characterizes the device state of the data acquisition device or the environmental information of the environment where the device acquisition device is located. The response module is configured to use a large language model to process the processed data and the reference data to obtain a response message for the input information; wherein, the large language model is deployed locally on an edge device based on model resources.

[0007] According to another aspect of the present disclosure, an application resource deployment device is provided, including: an acquisition module, a matching relationship determination module, and a second sending module. The acquisition module is configured to acquire device information from multiple edge devices in an edge device. The matching relationship determination module is configured to determine a matching relationship between the multiple edge devices and multiple application resources according to the device information of the multiple edge devices and demand information, where the multiple application resources include model resources. The second sending module is configured to, based on the matching relationship, send resource information of the matching application resources to the multiple edge devices, so that the edge device locally deploys the matching application resources and executes the above response method based on the matching application resources.

[0008] According to another aspect of the present disclosure, an electronic device is provided, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method provided by the present disclosure.

[0009] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, where the computer instructions are used to cause a computer to execute the method provided by the present disclosure.

[0010] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, where the computer program implements the method provided by the present disclosure when executed by a processor.

[0011] According to another aspect of the present disclosure, an information processing system is provided, including: a cloud, an edge device, and a terminal device. The cloud is configured to execute the above application resource deployment method; the edge device is configured to execute the above response method; the terminal device is configured to send input information to the edge device and receive a response message from the edge device.

[0012] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood from the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The drawings are used to better understand the present solution and do not constitute a limitation to the present disclosure. Among them:

[0014] Figure 1 is a schematic diagram of the application scenario of the information processing system according to an embodiment of the present disclosure;

[0015] Figure 2 is a schematic flowchart of the response method according to an embodiment of the present disclosure;

[0016] Figure 3 is a schematic principle diagram of the predetermined processing mode according to an embodiment of the present disclosure;

[0017] Figure 4 is a schematic flowchart of the application resource deployment method according to an embodiment of the present disclosure;

[0018] Figure 5 is a schematic structural diagram of the information processing system according to an embodiment of the present disclosure;

[0019] Figure 6 is a schematic structural diagram of the information processing system according to another embodiment of the present disclosure;

[0020] Figure 7 is a schematic block diagram of the structure of the response device according to an embodiment of the present disclosure;

[0021] Figure 8 is a schematic block diagram of the structure of the application resource deployment device according to an embodiment of the present disclosure; and

[0022] Figure 9 is a block diagram of the structure of the electronic device for implementing the response method and / or the application resource deployment method according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0023] The following describes exemplary embodiments of the present disclosure with reference to the drawings. Various details of the embodiments of the present disclosure are included to facilitate understanding and should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, descriptions of well-known functions and structures are omitted below for clarity and conciseness.

[0024] In some embodiments, the model can be deployed in the cloud and used to process the information input by the user.

[0025] With the above technical solution, since the model is deployed in the cloud, the communication latency between the user's terminal device and the cloud is relatively large in the case of a weak network, which cannot meet the real-time requirements. In response to this, in the embodiments of the present disclosure, the large language model is deployed in the edge device, so that the effect of local low-latency and fast response can be achieved by using edge computing technology, and unnecessary uplink and downlink data transmission can be reduced, thereby improving the real-time performance of the response information fed back to the user.

[0026] In addition, the modal requirements of users for input information and response information vary, and the tasks processed by different models are generally different. For example, some models can process text, and some models can process images. A single model cannot meet the multi-modal requirements. Therefore, multiple models need to be deployed in the cloud. In response to this, in the embodiments of the present disclosure, a large language model is adopted, which can process multi-modal input information such as text, speech, and images, and can also output multi-modal response information to the user (such as generating more intelligent text answers, voice prompts, image recommendations, and other forms of response information), thereby meeting the rich media interaction requirements and the preferences and needs of different users.

[0027] In addition, in some application scenarios, the answers and solutions to user questions need to integrate multi-modal data from external devices and sensors, while the model generally can only rely on the data during model training to respond and cannot effectively integrate local device data. In this case, the intelligence of the response information output by the model is relatively low. In response to this, the embodiments of the present disclosure provide reference information such as the device status and environmental conditions collected by the terminal device for the large language model, so as to obtain more intelligent and comprehensive output information.

[0028] In summary, the embodiments of the present disclosure aim to provide a response method that combines edge computing with a large language model and provides reference information collected by the acquisition device for the large language model, thereby allowing users to interact with the edge device in multiple input modalities such as voice, text, and image, and can determine non-customized and adaptive response information on the edge device based on the user's rich media input, the reference information of the acquisition device, and the context, achieving the effect of real-time adaptive multi-modal response.

[0029] The application scenarios applicable to the present disclosure include smart home, smart factory, autonomous driving, smart healthcare, etc.

[0030] Figure 1 It is a schematic diagram of the application scenario of the information processing system according to the embodiments of the present disclosure.

[0031] It should be noted that Figure 1 The figure shown is only an example of the system architecture to which the embodiments of the present disclosure can be applied, to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios.

[0032] As Figure 1 shown, the system architecture 100 according to this embodiment may include a cloud 101, edge devices 102, terminal devices 1031, 1032, 1033, and acquisition devices 1041, 1042. Each device communicates with each other through a network, and the network may include various connection types, such as wired and / or wireless communication links, etc.

[0033] The cloud 101 may be a server that provides various services. The cloud 101 may configure multiple functional modules required by the edge devices 102 and send them to the edge devices 102.

[0034] The edge devices 102 may include only a single device or may include a cluster composed of multiple devices. The edge devices include multiple application resources, such as large language models, etc.

[0035] Users can use the terminal devices 1031, 1032, 1033 to interact with the edge devices 102 through the network to send input information to the edge devices 102 or receive response information. The terminal devices 1031, 1032, 1033 may be various electronic devices with a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers, etc.

[0036] The acquisition devices 1041, 1042 may include temperature sensors, humidity sensors, cameras, vehicle-mounted devices, smart home devices, etc. Through the acquisition devices, the device's own status or environmental information can be acquired, and this reference information is sent to the edge devices 102 so that the edge devices 102 can determine the response information according to the user's input information and the reference information of the acquisition devices.

[0037] It should be noted that the response method provided by the embodiments of the present disclosure is generally executed by the edge devices 102. Correspondingly, the response device provided by the embodiments of the present disclosure is generally set in the edge devices 102.

[0038] It should be understood that Figure 1 the numbers of the cloud 101, edge devices 102, terminal devices 1031, 1032, 1033, and acquisition devices 1041, 1042 in

[0039] Figure 2 are merely illustrative. According to the implementation requirements, there may be any number of clouds, edge devices, terminal devices, and acquisition devices.

[0040] As Figure 2As shown, the response method 200 may include operation S210 to operation S230, and the response method 200 may be executed by an edge device.

[0041] In operation S210, in response to receiving input information of a target object, according to the modality of the input information, the input information is processed using a predetermined processing mode corresponding to the modality to obtain processed data.

[0042] In operation S220, reference data collected by a data acquisition device is determined.

[0043] In operation S230, the processed data and the reference data are processed using a large language model (LLM) to obtain a response message for the input information, where the large language model is locally deployed on the edge device based on model resources transmitted from the cloud.

[0044] For example, the target object may be a user with interaction requirements. For example, the modality of the input information may include at least one of text, voice, image, and video. The predetermined processing mode represents what kind of processing is performed on the input information. The mapping relationship between the modality of the input information and the predetermined processing mode can be pre-configured, and based on the mapping relationship, it is determined what kind of processing is performed on the input information. The predetermined processing mode mainly converts the format of the input information so that the format of the input information conforms to the input format of the large language model.

[0045] In practical applications, the predetermined processing mode may only include multiple sub-processing processes, and the multiple sub-processing processes may be executed serially or in parallel, and the execution order of the multiple sub-processing processes is pre-configured. In some embodiments, the predetermined processing mode may also only include a single sub-processing process.

[0046] For example, the data acquisition device may include a temperature sensor, a humidity sensor, a smoke sensor, a pressure sensor, a sound sensor, a gas sensor, a liquid level sensor, a camera, a vehicle device, and a smart home device, etc., and may also include a photoelectric sensor for detecting whether there is an object, a touch sensor for detecting whether there is contact, and a magnetic sensor, etc.

[0047] The reference data collected by the data acquisition device may represent the environmental information of the environment where the device acquisition device is located. For example, the reference data may include temperature data, humidity data, smoke data, environmental pressure data (such as atmospheric pressure), sound data, gas concentration information (such as carbon dioxide concentration, vehicle exhaust concentration, liquid level height data of certain waters, image data for a predetermined area, etc.

[0048] The reference data collected by the data acquisition device can also characterize the state of the in-vehicle system. For example, the reference data includes vehicle data, and the vehicle data may include, for example, the set temperature of the vehicle air conditioner, driving speed, acceleration, attitude, distance, vibration information, volume level, seat angle, seat temperature, etc. It should be noted that the vehicle itself is equipped with some sensors to detect the vehicle state. For example, the vehicle is equipped with a speed sensor, an acceleration sensor, an attitude sensor, a distance sensor, a vibration sensor, etc. These sensors can be electrically connected to the vehicle's electronic control unit and send the collected data. Therefore, the sensors equipped on the vehicle can be used as the data acquisition device, and the corresponding reference data can be obtained based on these sensors, or the above reference data can be directly obtained from the vehicle's electronic control unit.

[0049] The reference data collected by the data acquisition device can also characterize the operating state of the smart home device. For example, the reference data may include the lamp switch state, air conditioner temperature, humidifier operation mode, water quality state after purification by the purifier, etc.

[0050] For example, the processed data and the reference data can be input into the large language model, and the large language model is used to generate the response information. In the actual processing process, the processed data and the reference data can be combined with a pre-configured predetermined prompt information template to obtain the prompt information (Prompt), and then the prompt information is input into the large language model, and the large language model outputs the response information.

[0051] In the embodiments of the present disclosure, the large language model is deployed in the edge device instead of in the cloud. Therefore, the communication delay between the terminal device and the edge device is small, ensuring the real-time requirement. At the same time, the large language model can support the input and output of multiple modal data, thus meeting the multi-modal requirement. In addition, the reference data collected by the acquisition device is used as part of the input of the large language model, providing more information for the large language model, so that the large language model can obtain a more intelligent and comprehensive response information.

[0052] This embodiment describes the predetermined processing mode.

[0053] In this embodiment, the predetermined processing mode includes a plurality of sub-processing modes and the processing order for the plurality of sub-processing modes. Correspondingly, after the edge device receives the input information of the target object, it can process the input information according to the modality of the input information, based on the plurality of sub-processing modes corresponding to the modality and the processing order for the plurality of sub-modes, to obtain the processed data.

[0054] For example, the predetermined processing for text may include sub-processing processes such as text cleaning, word segmentation, stop word removal, word embedding, padding, truncation, stemming, named entity recognition, entity relationship extraction, sentiment analysis, topic modeling, etc.

[0055] The predetermined processing for speech may include automatic speech recognition to obtain text data, and then the processing process for the text data may refer to the predetermined processing mode of the above text.

[0056] The predetermined processing for images may include sub-processing processes such as denoising, brightness adjustment, contrast adjustment, color correction, image scaling, image cropping, image segmentation, feature extraction, object recognition and classification, target tracking, and image post-processing.

[0057] The predetermined processing for videos may include frame sampling, frame processing, time series processing, optical flow analysis, etc.

[0058] In addition, the input information may also include data collected by sensors. Correspondingly, the predetermined processing for sensor data may include cleaning, timestamp alignment, feature engineering, data interpolation, numerical pre-computation, etc. Numerical pre-computation may include other calculations such as data offset superposition, magnification and reduction by multiples.

[0059] Taking the image in the input information as an example below, the predetermined processing mode will be described. In this embodiment, the image may be an image input by the user on the front-end interface of the terminal device, or an image collected based on sensors.

[0060] For example, after obtaining the image data, the image can be preprocessed. The preprocessing includes denoising, brightness adjustment, contrast adjustment, color correction, image scaling, and image cropping, etc. Denoising can remove the noise in the image, such as blurring or irrelevant pixels. Brightness and contrast adjustment can adjust the brightness and contrast of the image to ensure appropriate visualization effects. Color correction can correct the color of the image to eliminate color cast. Image scaling or cropping can scale or crop the image as needed.

[0061] After preprocessing, image segmentation can be performed. Image segmentation is used to divide the image into different regions or objects. Image segmentation can be achieved through methods such as edge detection, threshold segmentation, region growing, etc.

[0062] After image segmentation, objects or regions are obtained, and then feature extraction can be performed. Feature extraction involves identifying and extracting information about the objects or regions, such as shape, color, texture, etc. These features can be used for subsequent analysis or recognition.

[0063] After extracting the feature data, according to the requirements, the objects in the image can be recognized and classified. Machine learning or deep learning algorithms can be used to detect and recognize objects, faces, vehicles, etc.

[0064] After the identified object or area, the movement and trajectory of the target can be tracked as needed. Subsequently, various post-processing operations can also be applied according to the task requirements, such as removing artifacts, drawing marks, visualizing results, etc.

[0065] Through the above-mentioned predetermined processing mode, effective information can be extracted from the image or specific analysis can be performed, and then the processed data is input into the large language model.

[0066] The above details describe the predetermined processing mode for the image in the input information. In other embodiments, some processing can also be performed on the image in the reference data, and the processing method can be the same as the predetermined processing mode, and then the processed reference data is used as the input of the large language model.

[0067] Figure 3 It is a schematic diagram of the predetermined processing mode according to the embodiments of the present disclosure.

[0068] In this embodiment, multiple operation logics can be pre-configured. For example, multiple operators and multiple operation symbols are displayed through the front-end page or console of the edge device or other terminals, and the staff can perform operations such as dragging on the operators and operation symbols, and this operation will trigger a selection instruction. Subsequently, the edge device combines the target operator among the multiple operators and the target operation symbol among the multiple operation symbols into an operation logic according to the selection instruction in the order selected by the operator. It can also be the console or other terminals that perform the combination operation according to the selection instruction to obtain the operation logic and output the operation logic to the edge device. The operation logic can come from the manual configuration of the background staff on the front-end operation page, defining the specific processing process from the acquisition point to the output point. The operation logic can also come from the system preset. For a certain type of fixed business scenario, a preset operation logic is provided and introduced by default. In addition, during the configuration of the operation logic, the corresponding relationship between the operation logic and the type of the predetermined protocol and the type of the terminal device can also be configured.

[0069] For example, the operation logic can be an original value mapping, that is, the values before and after the mapping remain unchanged. Another example is that the operation logic can include logical processing such as filtering and comparison. For example, in the temperature pipe control scenario, a temperature filtering template can be default added to filter out the sub-data of the points where the temperature is higher or lower than the threshold, so as to ensure the reliability of the reference data. Another example is that the operation logic can include function processing, and processes such as amplification, calculation, and iteration are performed through function processing. New operation logics can also be obtained by repeating and combining the above-mentioned original value mapping, logical processing, and function processing in various ways.

[0070] Such as Figure 3As shown, the edge-side device can obtain the original acquisition data from the terminal device that is connected to the edge-side device based on a predetermined acquisition period or a received data acquisition instruction. The predetermined protocol can include Modbus, BACnet, IPC, OPC-UA, etc. Accordingly, the acquisition devices can be classified into Modbus devices 311, BACnet devices 312, IPC devices 313, etc. according to the protocol category. For example, the Modbus device 311 can include a temperature sensor, a smoke sensor, etc., and the BACnet device 312 can include a humidity sensor, etc.

[0071] The edge device can include modules such as a soft gateway 323, a driver, and a message middleware. Each acquisition device 311, 312, 313 is linked to the corresponding driver 312, 322, 323. Each acquisition device 311, 312, 313 acquires a number of points, and each point has a unique identifier, such as point 0, point 1, etc. The original acquisition data collected includes sub-data of multiple points. The sub-data of multiple points is encapsulated into a unified format message by the corresponding driver and then sent to the soft gateway 323.

[0072] The soft gateway 323 can determine the target operation logic from multiple candidate operation logics according to the correspondence between the configured operation logic and the type of the predetermined protocol and the type of the terminal device. Then, based on the target operation logic, relevant calculation mapping is performed on the sub-data of multiple points to obtain the output point data. For example, the output point data 3241, 3242, 3243 is obtained. After that, the output point data 3241, 3242, 324 can be encapsulated into the reference data 325 according to the unified message protocol format, and then the reference data is published to the message middleware module.

[0073] It should be noted that the difference from deploying the algorithm in the soft gateway of the edge device is that: the algorithm deployed in the soft gateway of the edge device usually writes the operation code and then downloads it to the soft gateway. If the operation method changes, the code needs to be rewritten, and the flexibility is poor. In this embodiment, the operation logic is used as a configuration rule and is downloaded to the edge-side device in the form of rule data or a rule configuration file. If the operation logic needs to be modified, by dragging operators and operation symbols on the front-end page and then downloading the operation logic again, the processing process of the sub-data of multiple points can be changed, and the flexibility is higher.

[0074] Figure 4 It is a schematic flowchart of an application resource deployment method according to an embodiment of the present disclosure.

[0075] As Figure 4 shown, the resource deployment method 400 can include operation S410 to operation S440. The resource deployment method 400 involves the cloud and the edge-side device.

[0076] In operation S410, multiple edge devices in the edge-side device respectively send their respective device information of the multiple edge devices to the cloud.

[0077] For example, the edge-side device adopts a cluster environment, that is, the edge-side device includes multiple edge devices. The device information of the edge devices includes the actual hardware information, the actual remaining resource amount, and the actual system type of each of the multiple edge devices. The actual hardware information may include the number of processor cores, the memory capacity, etc., mainly characterizing the computing power and storage capacity. The actual remaining resource amount may include the remaining storage capacity, etc., and the actual system type may include IOS, Android, Windows, Linux, etc.

[0078] In operation S420, the cloud determines the matching relationship between the multiple edge devices and the multiple application resources according to the device information and the demand information.

[0079] For example, the demand information may include demand hardware information, demand resource amount, and demand system type. For example, an edge device with a Windows system that requires a 4-core GPU and 1T of memory is needed.

[0080] For example, the application resources can represent the resources for implementing specific application functions. The application resources can be implemented by software, and a functional module can be used as an application resource. The multiple application resources may include model resources, and the model resources can be used to deploy large language models. For example, the multiple application resources may also include core modules, model services, soft gateways, drivers, message middleware modules, user interaction modules, rule engine modules, etc. Each module will be described below and will not be elaborated here.

[0081] For example, the multiple application resources can be arranged to obtain a sequence. Then, the application resources in the sequence are sequentially determined as the resources to be matched. For the application resources to be matched among the multiple application resources, the edge devices with the same actual hardware information and demand hardware information and the same actual system type and demand system type are determined as candidate devices. After that, when the number of candidate devices is determined to be at least two, the sorting of the at least two candidate devices is determined according to the actual remaining resource amount and demand resource amount of each of the at least two candidate devices. For example, during the sorting process, the candidate devices with an actual remaining resource amount less than the demand resource amount can be excluded. For the candidate devices with a demand resource amount greater than or equal to the actual remaining resource amount, if the difference between the demand resource amount and the actual remaining resource amount is smaller, the sorting of the candidate device is higher. Then, according to the sorting, the edge device that satisfies the matching relationship with the application resource to be matched is determined from the at least two candidate devices. For example, the candidate device with the highest sorting is selected and used as the edge device that matches the resource to be matched. Then, the next resource to be matched in the sequence is processed until an edge device is allocated for each resource to be matched.

[0082] In operation S430, the cloud sends the resource information of the matching application resources to multiple edge devices based on the matching relationship.

[0083] For example, the resource information can be the download address of the application resource, and the edge device can download the matching application resource based on the download address. Another example is that the resource information can also be the application resource itself.

[0084] In operation S440, multiple edge devices among the edge devices deploy the matching application resources locally on the edge devices according to the received resource information.

[0085] In this embodiment, each application resource can be deployed and started in a suitable edge device according to the requirement information and device information. For example, the core module can be started on a Windows personal host, the model service can be deployed on a server of a large Linux system, the user interaction module can be deployed on a mobile phone terminal, and the soft gateway and driver module can be deployed on the corresponding device acquisition board. By adopting this technical solution, the computing power of the edge devices can be utilized more fully. For example, some edge devices have strong computing power, so the model service is deployed on this edge device. Some edge devices have strong GPU image processing capabilities, so the module for executing the preprocessing mode to parse video processing pictures is deployed in this edge device. Therefore, the utilization efficiency of the edge devices is improved and the cost is reduced.

[0086] In addition, this embodiment can also be compatible with multiple architectures and platforms, can access multi-platform devices for linkage, and expand the support types of the edge devices. For example, the model service is deployed on a linux server, the user information interaction module is deployed on IOS, and the response information is output in the form of pictures or audio through an Android device, thereby improving system compatibility.

[0087] As Figure 5 shown, in this embodiment, the information processing system may include a cloud 510, edge devices 520, terminal devices 530, and acquisition devices 540.

[0088] During the working process, the cloud 510 and the edge devices 520 can be used to execute the above resource deployment method, so as to deploy each application resource to the local of the edge devices 520. The acquisition device 540 is used to send the acquired reference information to the edge devices 520. The edge devices 520 can also be used to execute the above response method, so as to process the input information input by the user and the reference information of the acquisition device 540, and output the response information. The terminal device 530 is used to send the input information to the edge devices 520 and receive the response information from the edge devices 520.

[0089] Figure 6It is a schematic structural diagram of an information processing system according to another embodiment of the present disclosure.

[0090] As Figure 6 shown, in this embodiment, the information processing system may include a console 650, a cloud 610, edge devices 620, collection devices 640, and terminal devices 630.

[0091] The console 650 is used to interact with a first object (such as a staff member). The first object can initiate requests to the cloud 610 through the console 650 to complete processes such as creating node resources, creating application resources, binding application resources to node resources, managing models, issuing commands, and receiving response data.

[0092] The cloud 610 may include a model management service and an edge computing cloud service. Among them, the model management service may include a model management module and a model training module. The model management module is responsible for functions such as model construction, optimization, and storage. The user's management of the model in the cloud 610 includes, but is not limited to, operations such as model optimization, model acceleration, model training iteration, model compression, model creation, deletion, modification, query, and model resource distribution. The edge computing cloud service may include service interfaces, node management, application management, configuration management, model management, and data synchronization services. Through the edge computing cloud service, node resource creation, node information management, model management, application resource generation, application resource management, command forwarding, and resource information synchronization with the edge side can be achieved.

[0093] The number of edge devices 620 can be multiple. Each edge device corresponds to a node resource of the cloud 610, so that the cloud 610 can manage each edge device 620 through the node resource. The edge device 620 can be a single-machine environment or a cluster composed of multiple homogeneous or heterogeneous devices and servers. The system of the device can be systems of different platforms, such as IOS, Android, Windows, Linux, etc. An edge device 620 may include a core module Core, a model service, a soft gateway, a driver, a message middleware module, a user interaction module, a rule engine module, etc. The core module is used to collect relevant information of the edge device and communicate with the cloud 610 regularly for resource data synchronization, communicate with the resource management center to submit resources for deployment, and communicate with the data storage part for data access and storage. The model service is used to obtain the input information of the user and the reference information of the collection device 640, analyze and disassemble them, and then publish the response information to the message middleware module. The soft gateway interacts with the collection device 640 through the driver, can report the device status and data information to the cloud 610, and can also perform reverse control on the collection device 640 according to instructions. The user interaction module is used to receive the input information of a second object (such as a user) and perform predetermined processing. Each of the other modules can execute the established business logic.

[0094] The acquisition device 640 may include a temperature sensor, a humidity sensor, a smoke sensor, a camera, a vehicle device, and a smart home device. The smart home device may include an air conditioner, a lamp, etc. Based on the acquisition device 640, temperature data, humidity data, smoke data, environmental pressure data collected by a pressure sensor, sound data collected by a sound sensor, gas concentration information collected by a gas sensor, liquid level height data collected by a liquid level sensor, image data for a predetermined area, etc. in the environment can be collected, and vehicle data and device information of the smart home device can also be obtained. Each acquisition device 640 may use the collected information as reference information and send it to the soft gateway through the corresponding driver.

[0095] The terminal device 630 may interact with a second object (such as a user) to obtain input information input by the user, and then send the input information to the user interaction module of the edge device 620.

[0096] The following Figure 6 , taking the process from establishing node resources to performing a user interaction as an example, illustrates the working process of the information processing system.

[0097] As Figure 6 shown, the first object may operate through the console 650 to create a node resource in the cloud 610 and create application resources required for the service, and associate the application resources with the node resource. Multiple application resources created by the first object in the cloud 610 may include, for example, a core module Core, a message middleware module, a rule engine module, etc. These application resources are used for end-cloud communication after deployment, obtaining the status of edge-side modules, communication between various modules on the device side, message routing, etc., and each application resource can be implemented by software. In addition, when the first object creates application resources in the cloud 610, it may specify the requirement information for deploying the application resources. The requirement information may include parameters such as the required system type, required resource quantity, required hardware information, etc.

[0098] The first object may select a suitable LLM generation model resource through the console 650 and associate the model resource with the node resource.

[0099] The first object may operate through the console 650 to select a suitable driver according to the actual device situation to complete the relevant configuration of the soft gateway, and associate the soft gateway and the driver with the node resource respectively. The driver applications available to the first object include applications encapsulated by general protocols provided by the edge computing framework, such as Modbus drivers, OPC-UA drivers, etc., and may also include driver applications developed and encapsulated by custom driver protocols.

[0100] The first object can obtain the node installation command through the console 650 and execute the installation in the edge device 620, so that the cloud 610 can send the core module Core, model service, soft gateway, driver, message middleware module, user interaction module, rule engine module, etc. associated with the node resources to the edge device 620, so that the edge device 620 can run these modules, where the large language model is deployed in the model service. During the actual installation process, the deployment of the installation calculation program in the edge device 620 can be completed in forms such as online installation, offline installation package, and image burning.

[0101] After the edge device 620 executes the installation command, the edge device 620 first starts the core module Core, and then the core module Core establishes a communication link with the cloud 610 for data synchronization. The application layer protocols used by the core module Core and the cloud 610 to establish a data channel include technologies such as websocket, http, mqtt, and http3.

[0102] The core module in the edge device 620 communicates with the cloud 610 periodically through the established link to collect and report local data and synchronize the application information of the cloud 610. The information collected by the core module includes the names, versions, and instance deployment status of all application modules, and the software and hardware information of all nodes in the cluster, such as container versions and node resource consumption status.

[0103] The core module in the edge device 620 downloads relevant resources according to the data synchronized by the cloud 610 and starts applications such as the model service, user interaction module, soft gateway, and driver.

[0104] The content synchronized by the core module from the cloud 610 includes the applications created by the first object and related configuration data, as well as the subsequent changes to the resources in the cloud 610, such as adding new application modules or configurations, updating existing application or configuration information, and deleting existing applications or configurations. In addition, when each module on the edge is deployed and started, when in a cluster environment composed of multiple server devices, it will first be scheduled according to the system type requirements and resource requirements specified in the first object, so that the application resources are deployed and started on appropriate nodes. If multiple edge devices meet the requirements, then the multiple candidate devices are sorted, and then the application resources are deployed in the edge device with the highest ranking.

[0105] After the soft gateway and driver are started, they will establish a connection with the corresponding acquisition device 640 according to the configuration. The driver can establish a link with the acquisition device 640 using Modbus, BACnet, etc., or it can also be a protocol driver customized by the first object.

[0106] The soft gateway obtains the reference information collected by the collection device 640 through the driver, and then uniformly encapsulates the reference information into a message format designed by the framework and publishes it to the message center module. The soft gateway obtains device information through the driver, which can be a pre-configured periodic task or can be collected according to the subscribed downlink control instructions. For the data transmitted between the modules on the edge side, a unified message format can be used for encapsulation. The message format may include the request identifier reqId, timestamp timestamp, metadata, protocol version version, processing method method, device data shadow, device reported attribute status shadow.reported, device expected attribute status shadow.desired, attribute status last updated time shadow.lastUpdated and other field information.

[0107] The second object interacts with the user interaction module on the edge side through dialogue and other forms. The second object interacts with the edge side device 620 in the form of voice, text, picture, video, etc. If the terminal device 630 has sensors such as temperature and humidity sensors and smoke sensors, the data of these sensors can also be input into the edge side device 620.

[0108] The user interaction module processes the received input information based on the predetermined processing mode to obtain processed data, and then publishes the processed data to the large language model through the message middleware module. The predetermined processing mode can be referred to above, and this embodiment will not be repeated here.

[0109] The large language model generates response information suitable for the local device environment based on the processed data and the reference information subscribed from the message center module. The large language model further processes the data including: syntactic and lexical analysis, sentiment analysis, topic modeling, sequence tagging, automatic task flow extraction, relationship extraction, subtask identification, task priority allocation, resource allocation, process review, operation implementation generation, task report analysis, anomaly detection and recovery, etc. The reference information subscribed by the large language model includes the state information of the acquisition device 640 itself, the external environment information collected by the acquisition device 640, etc.

[0110] The large language model publishes the response information to the user interaction module through the middleware module. The large language model sends the message to the soft gateway through the message middleware module or directly. The message can be published using multiple protocols such as MQTT, HTTP, WebScoket, GRPC, etc.

[0111] The user interaction module provides the response message to the second object. According to the actual situation, the user interaction module can interact with the second object in various forms such as text, pictures, videos, and voices. At the same time, it can further send instructions to other edge-connected devices and participate in the interaction with the second object.

[0112] It should be noted that in this embodiment, each functional module can be implemented by software. Each functional module is not limited to platforms and systems, supports multiple implementations, and can be cross-platform deployed to various devices.

[0113] The above process of creating a slave node resource once, receiving the input information of the user, and then analyzing based on the large language model and integrating the reference information of other acquisition devices 640 to obtain the response information has been described.

[0114] Figure 7 It is a schematic structural block diagram of a response device according to an embodiment of the present disclosure.

[0115] As Figure 7 shown, the response device 700 may include a processing module 710, a reference data determination module 720, and a response module 730.

[0116] The processing module 710 is configured to, in response to receiving the input information of the target object, process the input information according to the modality of the input information by using a predetermined processing mode corresponding to the modality, and obtain the processed data.

[0117] The reference data determination module 720 is configured to determine the reference data from the data acquisition device, where the reference data characterizes the device state of the data acquisition device or the environmental information of the environment where the device acquisition device is located.

[0118] The response module 730 is configured to process the processed data and the reference data by using a large language model to obtain a response information for the input information; wherein, the large language model is deployed locally on the edge-side device based on the model resources.

[0119] In this embodiment, the reference data determination module includes: an acquisition sub-module, a target logic determination sub-module, a first processing sub-module, and a reference data determination sub-module. The acquisition sub-module is configured to obtain the original acquisition data from the data acquisition device based on a predetermined acquisition period or a received acquisition instruction; wherein, the data acquisition device is connected to the edge-side device based on a predetermined protocol, and the original acquisition data includes sub-data of multiple points. The target logic determination sub-module is configured to determine the target operation logic from multiple candidate operation logics according to the type of the predetermined protocol and the type of the data acquisition device. The first processing sub-module is configured to process the sub-data of multiple points according to the target operation logic and the order of multiple points. The reference data determination sub-module is configured to determine the reference data according to the sub-data of multiple points processed by the target operation logic.

[0120] In this embodiment, the candidate operation logic is obtained through an output module and a combination module. The output module is used to output a plurality of operators and a plurality of operation symbols. The combination module is used to respond to the received selection instruction, and according to the selection instruction, combine the target operator among the plurality of operators and the target operation symbol among the plurality of operation symbols into the candidate operation logic.

[0121] In this embodiment, the processing module includes: a second processing sub-module, which is used to process the input information according to a plurality of sub-processing modes corresponding to the modality of the input information and the processing sequence for the plurality of sub-processing modes, so as to obtain the processed data.

[0122] In this embodiment, the reference data collected by the data acquisition device includes at least one of the following: temperature data collected by a temperature sensor, humidity data collected by a humidity sensor, smoke data collected by a smoke sensor, ambient pressure data collected by a pressure sensor, sound data collected by a sound sensor, gas concentration information collected by a gas sensor, liquid level height data collected by a liquid level sensor, image data for a predetermined area collected by a camera, vehicle data collected by a vehicle-mounted device, and device information of a smart home device.

[0123] In this embodiment, the edge-side device includes a plurality of edge devices, and the apparatus further includes: a first sending module and a deployment module. The first sending module is used to send the device information of each of the plurality of edge devices to the cloud, so that the cloud determines the matching relationship between the plurality of edge devices and a plurality of application resources according to the device information and the requirement information, and sends the resource information of the matching application resources to the plurality of edge devices based on the matching relationship. The deployment module is used to respond to the received resource information, and according to the resource information, locally deploy the matching application resources on the edge devices, and the matching application resources include model resources.

[0124] Figure 8 It is a schematic structural block diagram of an application resource deployment apparatus according to an embodiment of the present disclosure.

[0125] As Figure 8 shown, the application resource deployment apparatus 800 may include an acquisition module 810, a matching relationship determination module 820, and a second sending module 830.

[0126] The acquisition module 810 is used to acquire the device information of a plurality of edge devices from the edge-side device.

[0127] The matching relationship determination module 820 is used to determine the matching relationship between the plurality of edge devices and the plurality of application resources according to the device information of the plurality of edge devices and the requirement information, and the plurality of application resources include model resources.

[0128] The second sending module 830 is configured to send resource information of the matching application resources to multiple edge devices based on the matching relationship, so that the edge devices can locally deploy the matching application resources and execute the above-mentioned response method based on the matching application resources.

[0129] In this embodiment, the device information of multiple edge devices includes the respective actual hardware information, actual remaining resource amounts, and actual system types of the multiple edge devices; the requirement information includes required hardware information, required resource amounts, and required system types; the matching relationship determination module includes: a candidate device determination sub-module, a sorting sub-module, and an edge device determination sub-module. The candidate device determination sub-module is configured to, for the application resource to be matched among multiple application resources, determine, as candidate devices, the edge devices whose actual hardware information is the same as the required hardware information and whose actual system type is the same as the required system type. The sorting sub-module is configured to, when the number of determined candidate devices is at least two, determine the sorting of the at least two candidate devices according to the respective actual remaining resource amounts and required resource amounts of the at least two candidate devices. The edge device determination sub-module is configured to determine, according to the sorting, the edge devices that satisfy the matching relationship with the application resource to be matched from the at least two candidate devices.

[0130] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, including at least one processor; and a memory communicatively connected to the at least one processor; the memory stores instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor is enabled to execute any of the above methods.

[0131] According to an embodiment of the present disclosure, the present disclosure also provides a non-transitory computer-readable storage medium storing computer instructions, where the computer instructions are used to cause a computer to execute any of the above methods.

[0132] According to an embodiment of the present disclosure, the present disclosure also provides a computer program product, including a computer program, where the computer program implements any of the above methods when executed by a processor.

[0133] Figure 9 It is a structural block diagram of an electronic device for implementing the response method and / or application resource deployment method of the embodiments of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, a personal digital processor, a cellular phone, a smart phone, a wearable device, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are only examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0134] As Figure 9 shown, device 900 includes a computing unit 901, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 902 or a computer program loaded from a storage unit 908 into a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the device 900 can also be stored. The computing unit 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0135] A plurality of components in the device 900 are connected to the I / O interface 905, including: an input unit 906, such as a keyboard, a mouse, etc.; an output unit 907, such as various types of displays, speakers, etc.; a storage unit 908, such as a magnetic disk, an optical disc, etc.; and a communication unit 909, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 909 allows the device 900 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0136] The computing unit 901 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 901 executes the various methods and processes described above, such as any of the above methods. For example, in some embodiments, any method can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as the storage unit 908. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 900 via the ROM 902 and / or the communication unit 909. When the computer program is loaded into the RAM 903 and executed by the computing unit 901, one or more steps of any of the methods described above can be executed. Alternatively, in other embodiments, the computing unit 901 can be configured to execute any method by any other appropriate means (e.g., by means of firmware).

[0137] The various embodiments of the systems and techniques described above in this specification can be implemented in digital electronic circuitry, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be a special-purpose or general-purpose programmable processor that receives data and instructions from, and transmits data and instructions to, a storage system, at least one input device, and at least one output device.

[0138] The program code for implementing the methods of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing device, such that the program codes, when executed by the processor or controller, cause the functions / operations specified in the flowchart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine, or entirely on the remote machine or server.

[0139] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0140] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, speech input, or tactile input).

[0141] The systems and techniques described herein can be implemented in a computing system including backend components (e.g., as a data server), or a computing system including middleware components (e.g., an application server), or a computing system including frontend components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system including any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), and the Internet.

[0142] A computer system can include a client and a server. The client and the server are generally far apart from each other and typically interact through a communication network. The relationship between the client and the server is generated by computer programs running on the respective computers and having a client-server relationship with each other.

[0143] It should be understood that various forms of the processes shown above can be used, with steps reordered, added, or deleted. For example, the steps recited in this disclosure can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this is not limited herein.

[0144] In the technical solutions of this disclosure, the collection, storage, use, processing, transmission, provision, and disclosure, etc. of the user's personal information involved all comply with the provisions of relevant laws and regulations and do not violate public order and good customs. In the technical solutions of this disclosure, prior to obtaining or collecting the user's personal information, the user's authorization or consent has been obtained. It should be noted that this disclosure emphasizes the privacy of user data and the security of device control, ensuring that information input by the user and information of the collection device are collected only when the user's authorization is obtained, thereby providing security guarantees.

[0145] The above specific embodiments do not constitute a limitation on the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principle of the present disclosure shall be included within the protection scope of the present disclosure.

Claims

1. A response method, comprising: In response to receiving input information of the target object, according to the modality of the input information, the input information is processed using a predetermined processing mode corresponding to the modality to obtain processed data; Determine reference data from a data acquisition device, wherein the reference data represents a device state of the data acquisition device or environmental information of an environment in which the data acquisition device is located; as well as The processed data and the reference data are processed using a large language model to obtain response information for the input information; wherein the large language model is deployed locally on an edge device based on model resources.

2. The method according to claim 1, wherein: Determining the reference data from the data acquisition device comprises: Based on a predetermined collection cycle or a received collection instruction, the original collection data from the data collection device is obtained; wherein the data collection device and the edge side device establish a connection based on a predetermined protocol, and the original collection data includes sub-data of multiple points; Determining a target operation logic from a plurality of candidate operation logics according to the type of the predetermined protocol and the type of the data acquisition device; Processing the sub-data of the plurality of points according to the target operation logic and the order of the plurality of points; and The reference data is determined according to the sub-data of the plurality of points processed by the target operation logic.

3. The method according to claim 2, wherein: The candidate operation logic is obtained in the following way: Output multiple operators and multiple operators; and In response to receiving a selection instruction, a target operator among the multiple operators and a target operator among the multiple operators are combined into the candidate operation logic according to the selection instruction.

4. The method according to claim 1, wherein: According to the modality of the input information, the input information is processed using a predetermined processing mode corresponding to the modality to obtain processed data including: The input information is processed according to a plurality of sub-processing modes corresponding to the modality of the input information and a processing order for the plurality of sub-processing modes to obtain processed data.

5. The method according to claim 1, wherein: The reference data collected by the data acquisition device includes at least one of the following: temperature data collected by a temperature sensor, humidity data collected by a humidity sensor, smoke data collected by a smoke sensor, ambient pressure data collected by a pressure sensor, sound data collected by a sound sensor, gas concentration information collected by a gas sensor, liquid level height data collected by a liquid level sensor, image data for a predetermined area collected by a camera, vehicle data collected by a vehicle computer, and device information of a smart home device.

6. The method according to claim 1, wherein: The edge side device includes a plurality of edge devices, and the method further includes: Sending device information of each of the plurality of edge devices to the cloud, so that the cloud determines a matching relationship between the plurality of edge devices and the plurality of application resources according to the device information and the demand information, and sends resource information of the matching application resources to the plurality of edge devices based on the matching relationship; and In response to receiving the resource information, matching application resources are deployed locally on the edge device according to the resource information, where the matching application resources include the model resources.

7. A method for deploying application resources, comprising: Obtain device information from multiple edge devices in the edge side device; Determine, according to the device information and demand information of the multiple edge devices, a matching relationship between the multiple edge devices and multiple application resources, wherein the multiple application resources include model resources; as well as Based on the matching relationship, resource information of the matched application resources is sent to the multiple edge devices, so that the edge devices can locally deploy the matched application resources, and execute the method described in any one of claims 1 to 6 based on the matched application resources.

8. The method according to claim 7, wherein: The device information of the plurality of edge devices includes actual hardware information, actual remaining resource amount and actual system type of each of the plurality of edge devices, and the demand information includes required hardware information, required resource amount and required system type; Determining the matching relationship between the plurality of edge devices and the plurality of application resources according to the device information and the demand information of the plurality of edge devices includes: For the application resources to be matched among the multiple application resources, an edge device whose actual hardware information is consistent with the required hardware information and whose actual system type is consistent with the required system type is determined as a candidate device; When it is determined that the number of the candidate devices is at least two, determining the order of the at least two candidate devices according to the actual remaining resource amount and the required resource amount of each of the at least two candidate devices; and According to the ranking, an edge device that satisfies the matching relationship with the application resource to be matched is determined from the at least two candidate devices.

9. A response device, comprising: A processing module, configured to, in response to receiving input information of a target object, process the input information according to a modality of the input information by adopting a predetermined processing mode corresponding to the modality to obtain processed data; A reference data determination module, used to determine reference data from a data acquisition device, wherein the reference data represents a device state of the data acquisition device or environmental information of an environment in which the data acquisition device is located; as well as A response module is used to process the processed data and the reference data using a large language model to obtain response information for the input information; wherein the large language model is deployed locally on the edge side device based on model resources.

10. The device according to claim 9, wherein: The reference data determination module comprises: An acquisition submodule, used to acquire original collected data from the data acquisition device based on a predetermined collection cycle or a received collection instruction; wherein the data acquisition device and the edge side device establish a connection based on a predetermined protocol, and the original collected data includes sub-data of multiple points; A target logic determination submodule, used to determine a target operation logic from a plurality of candidate operation logics according to the type of the predetermined protocol and the type of the data acquisition device; A first processing submodule, configured to process the sub-data of the plurality of points according to the target operation logic and the order of the plurality of points; and The reference data determination submodule is used to determine the reference data according to the sub-data of the multiple points processed by the target operation logic.

11. The device according to claim 10, wherein: The candidate operation logic is obtained through the following modules: Output module, used to output multiple operators and multiple operators; as well as A combination module is used for, in response to receiving a selection instruction, combining a target operator among the multiple operators and a target operator among the multiple operators into the candidate operation logic according to the selection instruction.

12. The device according to claim 9, wherein: The processing module comprises: The second processing submodule is used to process the input information according to a plurality of sub-processing modes corresponding to the modality of the input information and a processing order for the plurality of sub-processing modes to obtain processed data.

13. The device according to claim 9, wherein: The reference data collected by the data acquisition device includes at least one of the following: temperature data collected by a temperature sensor, humidity data collected by a humidity sensor, smoke data collected by a smoke sensor, ambient pressure data collected by a pressure sensor, sound data collected by a sound sensor, gas concentration information collected by a gas sensor, liquid level height data collected by a liquid level sensor, image data for a predetermined area collected by a camera, vehicle data collected by a vehicle computer, and device information of a smart home device.

14. The device according to claim 9, wherein: The edge side device includes a plurality of edge devices, and the apparatus further includes: a first sending module, configured to send device information of each of the plurality of edge devices to the cloud, so that the cloud determines a matching relationship between the plurality of edge devices and the plurality of application resources according to the device information and the demand information, and sends resource information of the matching application resources to the plurality of edge devices based on the matching relationship; and A deployment module is used to, in response to receiving resource information, deploy matching application resources locally on the edge device according to the resource information, wherein the matching application resources include the model resources.

15. An application resource deployment device, comprising: An acquisition module, used to acquire device information from multiple edge devices in the edge side device; A matching relationship determination module, configured to determine a matching relationship between the plurality of edge devices and a plurality of application resources according to device information and demand information of the plurality of edge devices, wherein the plurality of application resources include model resources; as well as The second sending module is used to send resource information of matching application resources to the multiple edge devices based on the matching relationship, so that the edge devices can locally deploy matching application resources and execute the method described in any one of claims 1 to 8 based on the matching application resources.

16. The device according to claim 15, wherein: The device information of the plurality of edge devices includes actual hardware information, actual remaining resource amount and actual system type of each of the plurality of edge devices, and the demand information includes required hardware information, required resource amount and required system type; The matching relationship determination module includes: A candidate device determination submodule, configured to determine, for the application resources to be matched among the multiple application resources, an edge device whose actual hardware information is consistent with the required hardware information and whose actual system type is consistent with the required system type as a candidate device; a sorting submodule, configured to, when it is determined that the number of the candidate devices is at least two, determine the sorting of the at least two candidate devices according to the actual remaining resource amounts and the required resource amounts of the at least two candidate devices respectively; and The edge device determination submodule is used to determine, according to the sorting, an edge device from the at least two candidate devices that satisfies the matching relationship with the application resource to be matched.

17. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 8.

18. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 8.

19. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 8.

20. An information processing system comprising: A cloud, configured to execute the method according to any one of claims 7 to 8; as well as The edge side device is configured to execute the method described in any one of claims 1 to 6.

Images