A multi-space intelligent machine-based coordinated reasoning system

Abstract

The application relates to the technical field of electric digital data processing, in particular to a coordination reasoning system based on multiple space intelligent machines. The system comprises at least one first central machine, any first central machine is connected with at least two space intelligent machines; when a user query question is received by a certain first central machine, the system executes the following steps: the first central machine decomposes the received question to obtain a plurality of sub-queries and respectively sends the sub-queries to corresponding space intelligent machines; the space intelligent machines process the received sub-queries and return query results and related memory data to the first central machine; the first central machine receives the query results and the memory data related to the received sub-queries returned by the space intelligent machines, combines the memory data related to the question stored in a context storage library to perform reasoning and generate a final answer corresponding to the question, and updates the context storage library. The application can improve the comprehensiveness of space perception and the accuracy of decision reasoning.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of electric digital data processing, in particular to a coordination reasoning system based on multi-space intelligent machines. BACKGROUND

[0002] With the development of Internet of Things and artificial intelligence technology, the demand for intelligent perception and decision-making of indoor environment is rapidly increasing, for example, long-distance task navigation planning of robots; for another example, providing space state query for intelligent applications and intelligent agents related to physical space; for another example, global real-time tracking and historical state query of personnel, equipment and assets in buildings. However, the existing technology is usually limited by the coverage range of its own sensors and computing power, and cannot efficiently and accurately process multi-target, multi-state tracking and reasoning tasks in complex environments, especially in large buildings. How to improve the comprehensiveness of space perception and the accuracy of decision-making reasoning is a problem to be solved. SUMMARY

[0003] The present application aims to provide a coordination reasoning system based on multi-space intelligent machines to improve the comprehensiveness of space perception and the accuracy of decision-making reasoning.

[0004] According to the present application, a coordination reasoning system based on multi-space intelligent machines is provided, the system comprising: at least one first central machine, any first central machine being connected with at least two distributedly deployed space intelligent machines, any space intelligent machine being equipped with sensors and cameras; any first central machine comprising a context repository and an interface for query.

[0005] When a certain first central machine receives a user query question, the system performs the following steps:

[0006] The first central machine decomposes the received question to obtain a plurality of sub-queries and sends them to the corresponding space intelligent machines respectively.

[0007] The space intelligent machines receive the sub-queries, process the received sub-queries, and return the query results and the memory data related to the received sub-queries to the first central machine;

[0008] The first central machine receives the query results and the memory data related to the received sub-queries returned by the space intelligent machines, combines the memory data related to the user's question stored in the context repository to perform reasoning and generate the final answer corresponding to the received question, and updates the context repository.

[0009] The present application has at least the following beneficial effects compared with the prior art:

[0010] The first hub machine in the multi-space intelligent machine-based coordinated reasoning system of the application can connect multiple space intelligent machines, maintain a context repository, and perform overall reasoning. When the first hub machine receives a user query question, it can generate a subquery for the space intelligent machine based on the user query question, and retrieve relevant existing memory content from the context repository; when the first hub machine receives the query result returned by the space intelligent machine and the memory data related to the received subquery, it can integrate the received query result and the memory data related to the received subquery with the relevant existing memory content retrieved from the context repository, perform reasoning through a large model, generate the final answer to the user question, and update the context repository.

[0011] The complete detailed information of the application is distributed in each space intelligent machine, and the global context of the hub machine only saves key states and associated memory data, so the bandwidth consumption is low; the hub machine and the space intelligent machine can perform their respective functions, and the independent models can be optimized respectively, so the computing power efficiency is high; the existence of the context repository makes the query, decision and reasoning more efficient and accurate, and avoids repeated query and data redundancy; based on this, the application can improve the comprehensiveness of space perception and the accuracy of decision reasoning. BRIEF DESCRIPTION OF DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the embodiments of the application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0013] Figure 1 The first flowchart of the steps performed by the multi-space intelligent machine-based coordinated reasoning system provided by the embodiments of the application;

[0014] Figure 2 The second flowchart of the steps performed by the multi-space intelligent machine-based coordinated reasoning system provided by the embodiments of the application. DETAILED DESCRIPTION

[0015] The technical solutions in the embodiments of the application will be described clearly and completely with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only some embodiments of the application, not all embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the application.

[0016] According to the embodiment, a multi-space intelligent machine-based coordinated reasoning system is provided, which comprises at least one first central machine, any first central machine being connected with at least two distributedly deployed space intelligent machines, any space intelligent machine being equipped with a sensor and a camera; any first central machine comprising a context repository and an interface for query.

[0017] In the embodiment, the space intelligent machine is an intelligent terminal equipped with a sensor and a camera and comprising a large model; any space intelligent machine is used to be responsible for local environment perception, data processing and real-time response, to realize understanding of the spatial structure of the corresponding physical place and tracking of the real-time state of the target object in the corresponding physical place. The central machine is a control unit comprising a large model, any central machine being capable of data integration, query decomposition and global reasoning based on the large model comprised thereby, the i-th central machine being a control unit of the i-th level of the system, i taking a value in the range of 1 to n, n being the number of levels of the central machines comprised by the system. Optionally, the large model is a trained large language model or an edge world model. As known by those skilled in the art, the large language model is a prior art, which will not be described herein. As an optional specific implementation, the edge world model is obtained on the basis of a preset world model in the cloud; specifically, the preset world model in the cloud is post-trained by the historical memory data stored by a certain space intelligent machine or central machine, to enable the preset world model to learn the unique features of the physical place corresponding to the space intelligent machine or central machine, and to obtain the edge world model fitting the physical place corresponding to the space intelligent machine or central machine by distilling the preset world model. As known by those skilled in the art, the distillation process is a prior art, which will not be described herein.

[0018] As a preferred specific implementation, the model distillation of the preset world model in the cloud based on the historical memory data stored by a certain space intelligent machine or central machine to obtain the edge world model fitting the physical place corresponding to the space intelligent machine or central machine comprises:

[0019] S1, obtaining the basic scene features corresponding to the target physical place according to the business scenario type corresponding to the target physical place.

[0020] In the embodiment, the target physical place is the physical place corresponding to any space intelligent machine or central machine.

[0021] In this embodiment, the basic scene features are key features extracted for a specific business scene type, which can reflect the basic scene and main attributes of the environment. The business scene type of any target physical site is known, and the basic scene features corresponding to each business scene type are also known. For example, when the business scene type is a smart factory, the basic scene features include equipment operating parameter ranges, key nodes of the production process, etc. When the business scene type is a smart home, the basic scene features include normal ranges of indoor temperature and humidity, common working modes of home appliances, etc.

[0022] Different business scene types have different characteristics and operating rules. By classifying and identifying the business scene types and extracting the corresponding basic scene features, targeted guidance can be provided for subsequent data processing and model training, so that the processing process is more in line with the actual situation of the target physical site, and the data utilization efficiency and model accuracy are improved.

[0023] S2, dimension reduction is performed on the historical memory data corresponding to the target physical site according to the basic scene features, to obtain the dimension-reduced historical memory data corresponding to the target physical site and the target scene features.

[0024] In this embodiment, the historical memory data corresponding to the target physical site may contain a large amount of redundant information and noise. High-dimensional data not only increases the computational cost, but also may affect the training effect of the model. By dimension reduction, the key information in the data can be highlighted, making the data more concise and easy to process. At the same time, dimension reduction combined with the basic scene features can retain important information related to the target physical site, so that the processed data is more in line with the needs of actual application.

[0025] As a specific implementation, S2 includes:

[0026] (1) For any basic scene feature corresponding to the target physical site, the proportion of the number of abnormal data in the historical memory data corresponding to the basic scene feature of the target physical site is obtained.

[0027] In this embodiment, for any basic scene feature, there is a corresponding preset data range. For any data in the historical data corresponding to the basic scene feature, it is compared with the corresponding preset data range. If it belongs to the corresponding preset data range, it is determined that the data in the historical data corresponding to the basic scene feature is normal data; otherwise, it is determined to be abnormal data.

[0028] In this embodiment, the ratio of the number of abnormal data in the historical memory data corresponding to any basic scene feature of the target physical site to the number of historical memory data corresponding to the basic scene feature is determined as the proportion of the number of abnormal data.

[0029] (2) Construct a difference distribution graph according to the proportion of the number of abnormal data in the historical memory data corresponding to each basic scene feature of the target physical site.

[0030] As a specific embodiment, the horizontal axis of the difference distribution graph represents the basic scene feature, and the vertical axis represents the proportion of the number of abnormal data in the corresponding historical memory data.

[0031] In this embodiment, the proportion of the number of abnormal data can represent the part deviating from the normal range in the historical memory data corresponding to the target physical site, and the proportion of the number of abnormal data often contains special information of the target physical site, such as abnormal situation, change trend, etc. The difference distribution graph directly shows the distribution of differences, which provides a basis for subsequent index screening.

[0032] (3) According to the difference distribution graph, the basic scene features corresponding to the target physical site are screened to obtain a plurality of target scene features.

[0033] In this embodiment, the basic scene features corresponding to the target physical site are screened according to the difference distribution graph. Specifically, this embodiment selects the scene features with a large proportion of the number of abnormal data (for example, greater than a certain preset proportion threshold) as target scene features. Since the selected target scene features can better reflect the uniqueness and change of the target physical site, they are more valuable for subsequent model training and analysis. For example, in the intelligent agricultural scene, if the proportion of the number of abnormal data of the soil humidity feature is large, the soil humidity feature is selected as the target scene feature.

[0034] (4) The memory data corresponding to the target scene features corresponding to the target physical site is determined as the reduced dimension historical memory data corresponding to the target physical site.

[0035] Based on (1)-(4), this embodiment realizes the reduction of data, removes some information that is not important for describing the characteristics of the target physical site, makes the data more refined and targeted, and thus improves the adaptability between the edge world model and the target physical site.

[0036] S3, the reduced dimension historical memory data and the target scene features corresponding to the target physical site are used as training data to perform model distillation on the preset world model in the cloud to obtain an edge world model corresponding to the target physical site.

[0037] In this embodiment, the reduced dimension historical memory data and the target scene features obtained after the reduction are integrated and input into the preset world model in the cloud as training data. Through model distillation on the preset world model, an edge world model that can accurately reflect the characteristics of the target physical site is finally obtained.

[0038] In this embodiment, the physical site refers to a physical space that needs to be processed and analyzed, such as a smart factory, a smart building, or a specific natural ecological area, which is the object range of data research and processing. The target object refers to an individual or component with independent characteristics and behaviors in the physical site, which can be a sensor, a device, a person, an object, a robot, etc. For example, in a smart factory, the target object can be a robot on the production line, various sensors, and transport vehicles.

[0039] For any hub machine or space intelligence machine, in order to ensure the independent data storage and tracking of each target object in its corresponding physical site, a unique ID corresponding to each target object in its corresponding physical site is generated, and a corresponding storage location is allocated to each ID in its corresponding memory, that is, a special area is demarcated for each ID in its corresponding memory to store the related data of the target object corresponding to each ID. The memory can be a database, a hard disk array, etc. In addition, the memory usually uses a specific data structure to manage the storage location, such as a hash table, an index tree, etc. Through the specific data structure, the corresponding storage location can be quickly found according to the ID, and the data storage, query, and update can be more efficient and accurate. Optionally, the ID can be a string of numbers, letters, or a combination of numbers and letters, which can be generated by a Universally Unique Identifier (UUID), Re-Identification (ReID), etc.

[0040] For any space intelligence machine, it stores the initial data of each target object in its corresponding physical site in each collection period, including collection data, vectorized features, and text summary data. The collection data of each target object is continuously collected by the sensors and cameras equipped by the space intelligence machine at a certain collection frequency, the vectorized features of each target object are obtained by vectorizing the collection data of each collection period of the target object, and the text summary data of each target object is obtained by text summarization of the collection data of each collection period of the target object. Those skilled in the art know that the process of vectorizing and text summarizing the collection data of sensors or cameras is prior art, which will not be described here. The initial data of this embodiment includes different dimensional data representation forms, which is beneficial to more comprehensively understand the state of each target object in each collection period in the physical site, and provides a rich data basis for deep mining and comprehensive utilization of data.

[0041] For any space intelligence machine, based on its stored initial data of each target object in its corresponding physical space for each collection period, a summary is performed every first preset time period. Optionally, for any space intelligence machine, the above summary is implemented according to a trained large model included therein, for example, the relevant data to be summarized and a preset prompt word are input into the trained large model, and the large model can output corresponding context data; it should be understood that the prompt word includes which summaries need to be performed by the large model (for example, summarizing data trends or summarizing device operating states or summarizing what happened in the physical space corresponding to the space intelligence machine within a period of time, etc.), and the preset prompt word can instruct the trained large model to perform a summary task on the input relevant data to be summarized. In actual applications, different prompt words can be set according to different needs.

[0042] For any hub machine, based on the preset type data uploaded by the space intelligence machines connected thereto, an induction is performed every second preset time period; wherein the preset type data uploaded by the space intelligence machines is pre-set, for example, including the results of the summary performed by the space intelligence machines every first preset time period, and optionally, the second preset time period is greater than the first preset time period; the physical space corresponding to any hub machine is composed of the physical spaces corresponding to all lower hub machines or space intelligence machines connected to the hub machine; it should be understood that the lower hub machine of the second hub machine is the first hub machine. Optionally, for any hub machine, the above induction is implemented according to a trained large model included therein, for example, the relevant data to be induced and a preset prompt word are input into the trained large model, and the large model can output corresponding induction results; it should be understood that the prompt word includes which inductions need to be performed by the large model, and the preset prompt word can instruct the trained large model to perform an induction task on the input relevant data to be induced. In actual applications, different prompt words can be set according to different needs. Wherein, induction refers to the text summary (summary) task in the traditional nlp field.

[0043] For any hub machine, the context repository included therein stores memory data of all target objects in its corresponding physical space, and the memory data includes global state and historical memory data. Wherein, the global state refers to the real-time state (such as position, action, state parameter, etc.) of each entity (person, object, device, etc.) in the space, and the historical memory data includes historical behavior records, feature vectors, sensor data, images, texts, etc. of each entity. The historical memory data stored by any hub machine can describe the state of the physical space corresponding to the hub machine within a historical time period and the state of each target object in the physical space, providing a data basis for the inference task performed by the large model included therein.

[0044] In this embodiment, when a first hub machine receives a question of a user query, the system performs the following steps, as shown in Figure 1

[0045] S100, the first hub machine decomposes the received question to obtain a plurality of sub-queries, and sends each sub-query to a corresponding spatial intelligent machine.

[0046] It should be understood that the first hub machine is the first hub machine that receives the question of the user query.

[0047] As an optional implementation, decomposing the received question to obtain a plurality of sub-queries includes:

[0048] S110, extracting a spatial entity in the question.

[0049] In this embodiment, the spatial entity in the question is the name of a physical site. As an optional implementation, a list of spatial entities is pre-set, which includes the names of the physical sites corresponding to each hub machine and spatial intelligent machine, and the spatial entity in the question is extracted by matching.

[0050] S120, if the spatial entity is successfully extracted, decomposing the extracted spatial entity according to a pre-set spatial topology map to obtain a plurality of sub-physical sites belonging to the spatial entity in the question; any sub-physical site corresponds to a spatial intelligent machine.

[0051] In this embodiment, if the question does not include a spatial entity, resulting in a failure to extract the spatial entity, the spatial entity is determined to be the physical site corresponding to the first hub machine.

[0052] In this embodiment, the spatial topology map is pre-constructed, which includes the physical sites corresponding to all hub machines included in the system and the physical sites included in all spatial intelligent machines, and also includes the attribution relationship between the physical sites, such as the attribution relationship of building-floor-room. In the pre-set spatial topology map, the physical site corresponding to the spatial entity in the question is located, and the lower-level physical sites included in the located physical site are determined as the sub-physical sites belonging to the spatial entity in the question.

[0053] S130, constructing a sub-query according to any sub-physical site and the original query content; the original query content is other content in the question except the spatial entity.

[0054] It should be understood that constructing a sub-query according to any sub-physical site and the original query content also means replacing the spatial entity in the question with the name of the sub-physical site.

[0055] ​As a specific implementation, the physical site corresponding to a certain first hub machine is the first floor of a building, the physical sites corresponding to the m space intelligent machines connected by the first hub machine are m rooms included in the first floor of the building; the question of the user query received by the first hub machine is: whether the temperature of the first floor is abnormal? Then, the first hub machine will decompose the question of the user query into: whether the temperature of the first room is abnormal, whether the temperature of the second room is abnormal, …, whether the temperature of the mth room is abnormal; and send them to the first, second, …, m space intelligent machines connected by the first hub machine respectively.

[0056] In this embodiment, if the constructed sub-query does not involve a certain space intelligent machine, the sub-query is not sent to the space intelligent machine. For example, the constructed sub-query only includes whether the temperature of the first room is abnormal and whether the temperature of the third room is abnormal, then only the sub-query of whether the temperature of the first room is abnormal is sent to the space intelligent machine corresponding to the first room, and the sub-query of whether the temperature of the third room is abnormal is sent to the space intelligent machine corresponding to the third room, and the sub-query is not sent to the space intelligent machine corresponding to the second space.

[0057] Based on S110-S130, the embodiment can decompose the question received by the first hub machine into several sub-queries.

[0058] S200, the space intelligent machine receives the sub-query, processes the received sub-query, and returns the query result and the memory data related to the received sub-query to the first hub machine.

[0059] As an optional specific implementation, for any space intelligent machine, if the space intelligent machine receives a sub-query, the memory data related to the sub-query is first obtained, and the memory data and the sub-query are input into the trained large model included in the space intelligent machine to obtain the query result. Optionally, the process of obtaining the memory data related to the sub-query includes: extracting the preset type entity in the sub-query, and searching in the memory data of the space intelligent machine according to the extracted preset type entity, and determining the searched memory data as the memory data related to the sub-query; optionally, the preset type entity includes a time entity and a collection parameter entity. For example, the sub-query received by a certain space intelligent machine is: whether the temperature of the first room in this week is abnormal, then this week is the time entity extracted from the sub-query, and the temperature is the collection parameter entity extracted from the sub-query.

[0060] As an optional specific implementation, the space intelligent machine receives the sub-query, and processing the received sub-query includes:

[0061] S210, extracting the collection parameter in the sub-query.

[0062] As an optional implementation, a list of collection parameters is preset, which includes the names of the parameters collected by the sensors and cameras equipped in the space intelligent machine, and the collection parameters in the sub-query are extracted by matching.

[0063] S220, if the extraction is successful, the extracted collection parameters are recorded as target collection parameters; if the extraction fails, the sub-query, the use property attribute of the physical site corresponding to the space intelligent machine, and the preset first prompt word are input into the large model of the space intelligent machine, and the collection parameters output by the large model are determined as the target collection parameters.

[0064] As a preferred implementation, the preset first prompt word is used to indicate that the large model outputs the collection parameters related to the sub-query. For example, the first prompt word includes: which collection parameters are related to the input sub-query? Or, the first prompt word includes: which collection parameters need to be known to answer the input sub-query? Those skilled in the art know that the first prompt word that can be used to indicate that the large model outputs the collection parameters related to the sub-query can be preset according to the needs. Optionally, the use property attribute of the physical site is used to represent the use function of the physical site, and the use property attribute of the physical site can be a hospital, a school, a shopping mall, a factory, a residence, an office building, etc., and the use property attribute of any physical site in this embodiment is known. For example, the sub-query is: environmental abnormality early warning; when the preset attribute type of the physical site corresponding to the space intelligent machine is different, the collection parameters output by the large model are also different. Based on the preferred implementation, the large model can determine the business scenario through the input use property attribute of the physical site, so that the large model can dynamically adjust the semantic understanding direction of the collection parameters according to the actual business scenario needs, avoid the inaccuracy of general parameter extraction in vertical scenarios, improve the scene adaptation capability of the space intelligent machine, and help to improve the accuracy of subsequent query results of the sub-query.

[0065] S230, if the space intelligent machine is not equipped with sensors and cameras that can directly collect the target collection parameters, the collection data of the target collection parameters is obtained according to the collection data of the sensors and cameras equipped in the space intelligent machine and associated with the target collection parameters.

[0066] It should be understood that if the target collection parameter is temperature, the sensor and camera that can directly collect the target collection parameter are sensors or cameras with temperature collection function.

[0067] In this embodiment, it is known which parameters can be directly monitored by sensors and cameras, and it is also known which parameters can be indirectly monitored by sensors and cameras. In the case where a spatial intelligent machine is not equipped with sensors and cameras for directly collecting the target collection parameters, the spatial intelligent machine can infer the collection data of the target collection parameters according to the collection data of the sensors and cameras associated with the target collection parameters equipped by the spatial intelligent machine, thereby improving the flexibility of obtaining query results and avoiding the situation that no query results can be given.

[0068] As an optional implementation, a large model included in the spatial intelligent machine is used to implement the function of obtaining the collection data of the target collection parameters according to the collection data of the sensors and cameras associated with the target collection parameters equipped by the spatial intelligent machine. For example, the collection data of the sensors and cameras associated with the target collection parameters equipped by the spatial intelligent machine and the task requirement of obtaining the collection data of the target collection parameters according to the collection data of the sensors and cameras associated with the target collection parameters equipped by the spatial intelligent machine are input to the large model, and then the output of the large model is the collection data of the target collection parameters.

[0069] In this embodiment, if the spatial intelligent machine is equipped with sensors and cameras for directly collecting the target collection parameters, the collection data of the target collection parameters is obtained.

[0070] S240, the collection data of the target collection parameters and the sub-query are input to the large model, and the output of the large model of the spatial intelligent machine is determined as the query result corresponding to the sub-query.

[0071] Based on S210-S240, the processing of the sub-query can be implemented to obtain the query result corresponding to the sub-query.

[0072] S300, the first hub machine receives the query result returned by the spatial intelligent machine and the memory data related to the received sub-query, combines the memory data related to the user's question stored in the context repository, infers and generates the final answer corresponding to the received question, and updates the context repository.

[0073] The first hub machine of this embodiment can integrate the query result returned from the spatial intelligent machine and the memory data related to the received sub-query and the memory data related to the user's question stored in the context repository, and infer through the large model included therein to generate the overall answer (i.e. the final answer) corresponding to the question, and update the context repository.

[0074] As an optional implementation, the query result returned from the spatial intelligent machine, the memory data related to the received sub-query, the memory data related to the user's question stored in the context repository and the first hub machine input the received question into the large model included in the first hub machine, determine the output of the large model as the overall answer to the question, and store the overall answer, the memory data related to the received sub-query and the question received by the first hub machine into the context repository, thereby updating the context repository of the first hub machine.

[0075] Based on S100-S300, the embodiment can quickly and accurately obtain the answer to the user's question.

[0076] As an optional implementation, the number of the first hub machines is greater than or equal to 2, and the system further includes at least one second hub machine, any second hub machine is connected with at least two first hub machines; any second hub machine includes a context repository and an interface for querying.

[0077] When a certain second hub machine receives a user query question, the system performs the following steps, as shown in Figure 2

[0078] S10, the second hub machine decomposes the received question to obtain a plurality of first queries and sends them to the corresponding first hub machines.

[0079] It should be understood that the second hub machine is the second hub machine that receives the user query question.

[0080] As an optional implementation, decomposing the received question to obtain a plurality of first queries includes:

[0081] S11, extracting the spatial entity in the question.

[0082] S12, if the spatial entity is successfully extracted, decomposing the extracted spatial entity according to a preset spatial topology map to obtain a plurality of first physical sites belonging to the spatial entity in the question; any first physical site corresponds to a first hub machine.

[0083] In the embodiment, if the question does not include a spatial entity, resulting in a failure to extract the spatial entity, the spatial entity is determined as the physical site corresponding to the second hub machine.

[0084] S13, constructing a first query according to any first physical site and the original query content; the original query content is other content in the question except the spatial entity.

[0085] ​In this embodiment, the space topology graph is pre-constructed, which includes the physical sites corresponding to all hub machines included in the system and the physical sites included in all space intelligent machines, and also includes the belonging relationships between the physical sites.

[0086] As a specific implementation, the physical site corresponding to a second hub machine is a building, the physical sites corresponding to the q first hub machines connected to the second hub machine are q floors included in the building; the question of the user query received by the second hub machine is: whether the temperature is abnormal? Then, the second hub machine will decompose the question of the user query into: whether the temperature of the first floor is abnormal, whether the temperature of the second floor is abnormal, …, whether the temperature of the qth floor is abnormal; and send to the first, second, …, q first hub machines connected to the second hub machine respectively.

[0087] Based on S11-S13, the second hub machine can decompose the received question into a plurality of first queries; the process of decomposing the received question into a plurality of first queries by the second hub machine is similar to S110-S130 described above, and will not be described here.

[0088] S20, the first hub machine receives the first query, decomposes the received first query to obtain a plurality of sub-queries, and sends to the corresponding space intelligent machine respectively.

[0089] As an optional specific implementation, the first hub machine receives the first query, decomposes the received first query to obtain a plurality of sub-queries, which includes:

[0090] S21, the first hub machine obtains the sub-physical sites included in the physical site corresponding to itself according to the pre-set space topology graph; any sub-physical site corresponds to a space intelligent machine.

[0091] In this embodiment, the space topology graph is pre-constructed, which includes the physical sites corresponding to all hub machines included in the system and the physical sites included in all space intelligent machines, and also includes the belonging relationships between the physical sites.

[0092] S22, constructing a sub-query according to each sub-physical site and the original query content.

[0093] Based on S21-S22, the first hub machine can decompose the received first query into a plurality of sub-queries. The process of decomposing the received first query into a plurality of sub-queries by the first hub machine is similar to S110-S130 described above, and will not be described here.

[0094] S30, the space intelligent machine receives the sub-query obtained by decomposing the first query, processes the received sub-query obtained by decomposing the first query, and returns the query result and the memory data related to the received sub-query obtained by decomposing the first query to the connected first hub machine.

[0095] S40, the first hub machine receives the query result returned by the space intelligence machine and the memory data related to the received sub-query obtained by decomposing the first query, combines the memory data related to the received first query stored in the context repository to perform reasoning and generates a query result corresponding to the received first query, and returns the generated query result and the memory data related to the received first query to the connected second hub machine, and updates the context repository of the first hub machine.

[0096] As an optional embodiment, for any first hub machine receiving a first query, the query result returned from the space intelligence machine, the memory data related to the received sub-query, the memory data related to the received first query stored in the context repository, and the first query received by the first hub machine are input into a large model included in the first hub machine, the output of the large model is determined as the query result corresponding to the first query received by the first hub machine, and the query result, the memory data related to the received sub-query, and the first query received by the first hub machine are stored in the context repository to update the context repository of the first hub machine.

[0097] As an optional embodiment, for any first hub machine, the process of obtaining memory data related to the first query includes extracting a preset type entity in the first query, and searching in the memory data of the hub machine according to the extracted preset type entity, and determining the searched memory data as the memory data related to the first query; optionally, the preset type entity includes a time entity and a collection parameter entity. For example, the first query received by a certain hub machine is: whether the temperature of this week in the first layer is abnormal, then this week is the time entity extracted from the first query, and the temperature is the collection parameter entity extracted from the first query.

[0098] S50, the second hub machine receives the query result returned by the first hub machine and the memory data related to the received first query, combines the memory data related to the user's question stored in the context repository of the second hub machine to perform reasoning and generates a final answer corresponding to the received question, and updates the context repository of the second hub machine.

[0099] As an optional embodiment, for any second hub machine, the process of obtaining memory data related to the question includes extracting a preset type entity in the question, and searching in the memory data of the hub machine according to the extracted preset type entity, and determining the searched memory data as the memory data related to the question; optionally, the preset type entity includes a time entity and a collection parameter entity.

[0100] As an optional implementation, the query result returned from the first hub, the memory data related to the received first query, the memory data related to the user's question stored in the context repository and the second hub input the received question into the large model included in the second hub, determine the output of the large model as the overall answer to the question, and store the overall answer, the memory data related to the received first query and the question received by the second hub into the context repository, thereby updating the context repository of the second hub.

[0101] Based on S10-S50, the embodiment can quickly and accurately obtain the answer to the user's question.

[0102] As an optional implementation, the number of the second hubs is greater than or equal to 2, and the system further comprises at least one third hub, any third hub being connected with at least two second hubs; any third hub comprising a context repository and an interface for querying.

[0103] In this embodiment, when a third hub receives a question from a user, the third hub decomposes the received question into several second queries and sends them to corresponding second hubs respectively; the second hubs receive the second queries, decompose the received second queries into first queries and send them to corresponding first hubs respectively; the first hubs receive the first queries, decompose the received first queries into several sub-queries and send them to corresponding spatial intelligent machines respectively; the spatial intelligent machines receive the sub-queries, process the received sub-queries and return query results and memory data related to the received sub-queries to the connected first hubs; the first hubs receive the query results and memory data related to the received sub-queries returned by the spatial intelligent machines, infer and generate query results corresponding to the received first queries in combination with memory data related to the received first queries stored in the context storage of the first hub, return the generated query results and memory data related to the received first queries to the connected second hubs, and update the context storage of the first hub; the second hubs receive the query results and memory data related to the received first queries returned by the first hubs, infer and generate query results corresponding to the received second queries in combination with memory data related to the received second queries stored in the context storage of the second hub, return the generated query results and memory data related to the received second queries to the connected third hubs, and update the context storage of the second hub; the third hubs receive the query results and memory data related to the received second queries returned by the second hubs, infer and generate final answers corresponding to the received question in combination with memory data related to the received question stored in the context storage of the third hub, and update the context storage of the third hub.

[0104] As an optional implementation, the number of the third hubs is greater than or equal to 2, and the system further comprises at least one fourth hub, any fourth hub being connected with at least two third hubs; any fourth hub comprising a context storage and an interface for query.

[0105] In this embodiment, when a fourth hub receives a question queried by a user, the fourth hub decomposes the received question to obtain several third queries and sends them to corresponding third hubs respectively; the third hub performs a process similar to that performed by the third hub after receiving a question queried by a user, with the difference that the third hub receives query results returned by a second hub and memory data related to the received second query, combines memory data related to the received third query stored in the context repository of the third hub to perform reasoning and generate query results corresponding to the received third query, returns the generated query results and memory data related to the received third query to the connected fourth hub, and updates the context repository of the third hub; the fourth hub receives query results returned by the third hub and memory data related to the received third query, combines memory data related to the received question stored in the context repository of the fourth hub to perform reasoning and generate a final answer corresponding to the received question, and updates the context repository of the fourth hub.

[0106] As an optional implementation, the number of fourth hubs is greater than or equal to 2, and the system further comprises at least one fifth hub, any fifth hub being connected to at least two fourth hubs; any fifth hub comprising a context repository and an interface for queries.

[0107] In this embodiment, when a fifth hub receives a question queried by a user, the fifth hub decomposes the received question to obtain several fourth queries and sends them to corresponding fourth hubs respectively; the fourth hub performs a process similar to that performed by the fourth hub after receiving a question queried by a user, with the difference that the fourth hub receives query results returned by a third hub and memory data related to the received third query, combines memory data related to the received fourth query stored in the context repository of the fourth hub to perform reasoning and generate query results corresponding to the received fourth query, returns the generated query results and memory data related to the received fourth query to the connected fifth hub, and updates the context repository of the fourth hub; the fifth hub receives query results returned by the fourth hub and memory data related to the received fourth query, combines memory data related to the received question stored in the context repository of the fifth hub to perform reasoning and generate a final answer corresponding to the received question, and updates the context repository of the fifth hub.

[0108] As an optional implementation, the number of the fifth central machines is greater than or equal to 2, and the system further comprises at least one sixth central machine, any sixth central machine is connected with at least two fifth central machines; any sixth central machine comprises a context repository and an interface for querying.

[0109] In the embodiment, when a sixth central machine receives a question of a user query, the sixth central machine decomposes the received question to obtain several fifth queries and sends them to corresponding fifth central machines respectively; the process executed by the fifth central machine after receiving the fifth query is similar to the process executed by the fifth central machine after receiving the question of the user query, the difference is that the fifth central machine receives the query result returned by the fourth central machine and the memory data related to the received fourth query, combines the memory data related to the received fifth query stored in the context repository of the fifth central machine to perform reasoning and generate a query result corresponding to the received fifth query, returns the generated query result and the memory data related to the received fifth query to the connected sixth central machine, and updates the context repository of the fifth central machine; the sixth central machine receives the query result returned by the fifth central machine and the memory data related to the received fifth query, combines the memory data related to the received question stored in the context repository of the sixth central machine to perform reasoning and generate a final answer corresponding to the received question, and updates the context repository of the sixth central machine.

[0110] In this embodiment, the system includes n layers of hub machines and spatial intelligence machines, wherein the i-th layer of hub machines includes a plurality of i-th hub machines, i = 1, 2, …, n; when i ≥ 2, the i-th hub machine is connected to at least two (i-1)-th hub machines, and the physical site corresponding to the i-th hub machine is the combination of the physical sites corresponding to all (i-1)-th hub machines connected thereto; when the i-th hub machine receives a question (query), the i-th hub machine can decompose the received question (query) and send the decomposed query to the corresponding (i-1)-th hub machine, and correspondingly, the (i-1)-th hub machine also returns the query result and the memory data related to the received (i-1)-th query to the i-th hub machine, so that the i-th hub machine can reason based on the received query result, the memory data related to the received (i-1)-th query and the memory data related to the received question (query) stored in the context repository of the i-th hub machine. When i = 1, the i-th hub machine is connected to at least two spatial intelligence machines, and the physical site corresponding to the i-th hub machine is the combination of the physical sites corresponding to all spatial intelligence machines connected thereto; when the i-th hub machine receives a question (query), the i-th hub machine can decompose the received question (query) and send the decomposed query to the corresponding spatial intelligence machine, and correspondingly, the spatial intelligence machine also returns the query result and the memory data related to the received sub-query to the i-th hub machine, so that the i-th hub machine can reason based on the received query result, the memory data related to the received sub-query and the memory data related to the question stored in the context repository of the i-th hub machine.

[0111] In the multi-spatial intelligence machine-based coordinated reasoning system of this embodiment, each first hub machine can be connected to multiple spatial intelligence machines, can maintain a context repository and can perform overall reasoning. When the first hub machine receives a question queried by a user, the first hub machine can generate a sub-query for the spatial intelligence machine based on the question queried by the user, and can retrieve relevant existing memory content from the context repository; after the first hub machine receives the query result returned by the spatial intelligence machine and the memory data related to the received sub-query, the first hub machine can integrate the received query result, the memory data related to the received sub-query and the relevant existing memory content retrieved from the context repository, perform reasoning through a large model, generate a final answer to the question of the user and update the context repository.

[0112] In this embodiment, the complete detailed information is distributed and stored in each spatial intelligence machine, and the global context of the hub machine only stores key states and associated memory data, so that the bandwidth consumption is low; the hub machine and the spatial intelligence machine can perform their respective functions, and the independent models can be optimized respectively, so that the computing power efficiency is high; the existence of the context repository makes the query, decision and reasoning more efficient and accurate, and avoids repeated query and data redundancy; based on this, this embodiment can improve the comprehensiveness of spatial perception and the accuracy of decision reasoning.

[0113] While certain specific embodiments of the application have been described in detail herein for the purposes of exemplification, numerous other variations and modifications will be apparent to persons skilled in the art. Any and all such variations and modifications are within the scope of this application as defined in the appended claims.

Claims

1. A multi-space intelligent agent based coordinated reasoning system, characterized by, The system comprises at least one first hub machine, any first hub machine is connected with at least two distributed spatial intelligence machines, any spatial intelligence machine is equipped with sensors and cameras; any first hub machine comprises a context repository and an interface for querying; When a first hub machine receives a user query question, the system performs the following steps: The first hub machine decomposes the received question into several sub-queries and sends them to the corresponding spatial intelligence machines respectively; The spatial intelligence machine receives the sub-query, processes the received sub-query, and returns the query result and the memory data related to the received sub-query to the first hub machine; The first hub machine receives the query result and the memory data related to the received sub-query returned by the spatial intelligence machine, combines the memory data related to the user's question stored in the context repository to infer and generate the final answer corresponding to the received question, and updates the context repository; The number of first hub machines is greater than or equal to 2, and the system further comprises at least one second hub machine, any second hub machine is connected with at least two first hub machines; any second hub machine comprises a context repository and an interface for querying; When a second hub machine receives a user query question, the system performs the following steps: The second hub machine decomposes the received question into several first queries and sends them to the corresponding first hub machines respectively; The first hub machine receives the first query, decomposes the received first query into several sub-queries, and sends them to the corresponding spatial intelligence machines respectively; The spatial intelligence machine receives the sub-query obtained by decomposing the first query, processes the received sub-query obtained by decomposing the first query, and returns the query result and the memory data related to the received sub-query obtained by decomposing the first query to the connected first hub machine; The first hub machine receives the query result and the memory data related to the received sub-query obtained by decomposing the first query returned by the spatial intelligence machine, combines the memory data related to the received first query stored in the context repository to infer and generate the query result corresponding to the received first query, and returns the generated query result and the memory data related to the received first query to the connected second hub machine, and updates the context repository of the first hub machine; The second hub machine receives the query result and the memory data related to the received first query returned by the first hub machine, combines the memory data related to the user's question stored in the context repository of the second hub machine to infer and generate the final answer corresponding to the received question, and updates the context repository of the second hub machine.

2. The multi-space intelligent agent-based coordinated reasoning system of claim 1, wherein, The spatial intelligence machine receives the sub-query, processes the received sub-query, which comprises: Extracting the collection parameters in the question; If the extraction is successful, the extracted collection parameters are recorded as target collection parameters; if the extraction fails, the sub-query, the use property attribute of the physical site corresponding to the spatial intelligence machine, and the preset first prompt word are input into the large model of the spatial intelligence machine, and the collection parameters output by the large model are determined as the target collection parameters; If the space intelligent machine is not equipped with sensors and cameras for directly collecting the target collection parameter, collection data of the target collection parameter is acquired according to collection data of sensors and cameras associated with the target collection parameter and equipped on the space intelligent machine; The collection data of the target collection parameter and the sub-query are input into the large model, and the output of the large model of the space intelligent machine is determined as the query result corresponding to the sub-query.

3. The multi-space intelligent agent-based coordinated reasoning system of claim 1, wherein, The received question is decomposed to obtain a plurality of sub-queries, including: Extracting the spatial entity in the question; If the spatial entity is successfully extracted, the extracted spatial entity is decomposed according to a preset spatial topology map to obtain a plurality of sub-physical sites belonging to the spatial entity in the question; any sub-physical site corresponds to a space intelligent machine; A sub-query is constructed according to any sub-physical site and the original query content; the original query content is other content in the question except the spatial entity.

4. The multi-space intelligent agent-based coordinated reasoning system of claim 1, wherein, The received question is decomposed to obtain a plurality of first queries, including: Extracting the spatial entity in the question; If the spatial entity is successfully extracted, the extracted spatial entity is decomposed according to a preset spatial topology map to obtain a plurality of first physical sites belonging to the spatial entity in the question; any first physical site corresponds to a first hub machine; A first query is constructed according to any first physical site and the original query content; the original query content is other content in the question except the spatial entity.

5. The multi-space intelligent agent-based coordinated reasoning system of claim 4, wherein, The first hub machine receives the first query, and decomposes the received first query to obtain a plurality of sub-queries, including: The first hub machine acquires sub-physical sites included in the physical site corresponding to itself according to a preset spatial topology map; any sub-physical site corresponds to a space intelligent machine; A sub-query is constructed according to each sub-physical site and the original query content.

6. The multi-space intelligent agent-based coordinated reasoning system of claim 1, wherein, The number of the second hub machines is greater than or equal to 2, and the system further includes at least one third hub machine, any third hub machine being connected with at least two second hub machines; any third hub machine including a context storage library and an interface for query.

7. The multi-space intelligent agent-based coordinated reasoning system of claim 6, wherein, The number of the third hub machines is greater than or equal to 2, and the system further includes at least one fourth hub machine, any fourth hub machine being connected with at least two third hub machines; any fourth hub machine including a context storage library and an interface for query.

8. The multi-space intelligent agent-based coordinated reasoning system of claim 7, wherein, The number of the fourth hub machines is greater than or equal to 2, and the system further includes at least one fifth hub machine, any fifth hub machine being connected with at least two fourth hub machines; any fifth hub machine including a context storage library and an interface for query.

9. The multi-space intelligent agent-based coordinated reasoning system of claim 8, wherein, The number of the fifth hub machines is greater than or equal to 2, and the system further includes at least one sixth hub machine, any sixth hub machine being connected with at least two fifth hub machines; any sixth hub machine including a context storage library and an interface for query.

Images