Self-processing method based on perception of real world

Abstract

The invention relates to a self-processing method based on perception of a real world, and the method comprises the steps: carrying out the symbolization of multi-modal perception data and corresponding processing result data, which are respectively collected, in the steps of optimization training and self-processing, so as to form a word vector for training and a word vector for execution, which have natural semantics; the word vectors for training describe multi-modal sensing data and processing result data corresponding to the collected sensing data, and the word vectors for execution describe the multi-modal sensing data; further, the perception data of the multiple different modes are converted into word vectors with natural semantics, and the information amount contained in the word vectors obtained through symbolization is approximately equal to the sum of the information amount contained in the perception data of the multiple different modes and the information amount contained in the corresponding processing result data. According to the method, the optimization training complexity of the special large language model is reduced, the training efficiency and the response efficiency are improved, and the training cost and the implementation cost are reduced.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence neural network technology, and more specifically, to a self-processing method based on perceiving the real world. Background Art

[0002] The types, scales and applications of artificial intelligence neural networks in the prior art are mainly used to process single-modal inputs, or to process different-modal inputs in combination with models of different modalities. Regardless of whether single-modal inputs or different-modal inputs are processed, the input itself is processed, that is, the corresponding feature vectors are extracted, and then the model is trained. Therefore, in the prior art, the trained artificial intelligence model has a strong application scenario (such as text question and answer, voice question and answer, image recognition, etc. At the same time, an artificial intelligence model in different technical fields of an application scenario cannot be directly applied across technical fields), especially the input modality has serious limitations; and when it is necessary to process multiple different modal and interrelated inputs, it is necessary to set up multiple models for application scenarios corresponding to different modalities, and then associate the processing results output separately. It can be seen that the prior art has many shortcomings in achieving multi-modal processing, such as multiple model types, the need for separate training, the need for combined training, limitations in application scenarios, and limitations in input modalities.

[0003] In the prior art, the processing results output by the trained artificial intelligence model are limited by the training samples, that is, the training samples have a strong decisive effect on the processing results of the artificial intelligence model. Once the training samples are biased, it is easy to cause the trained model to have certain defects.

[0004] When the processing results of the artificial intelligence model based on the prior art are further applied, for example, the processing results are used as instructions to control other execution devices. Since the operation of the execution device is not directly controlled by the artificial intelligence model, but it is necessary to match the instructions of the execution device with various processing results, during implementation, the processing results are first output through the artificial intelligence model, and then the instructions that match the processing results are found, and the execution device then acts based on the instructions. The instructions used to control the execution device are fixed, which in turn limits the diversity of the processing results (that is, other situations in which the processing results have been matched are not allowed, and negation will not be used to control the execution device). At the same time, the output results of the artificial intelligence model are prone to security weaknesses and attacks when matching corresponding instructions.

[0005] With the "emergence" of large models, the limitations of artificial intelligence models in application scenarios have been overcome to a certain extent, especially the formation of general models under certain conditions, that is, under certain conditions, they can be applied to different technical fields. Furthermore, by combining large-scale language models with large-scale models of other modalities, a multimodal large-scale model can be obtained. A multimodal large-scale model can be used to process language and other modalities and generate multimodal outputs. For example, the PaLM-E model is a large-scale language model combined with a large-scale visual model. When deployed on a robot, requirements can be put forward through natural language, and then the robot automatically executes the requirements based on visual input.

[0006] However, when the large-scale multimodal model in the prior art processes multimodal input, each modal input is processed in its own modality. For example, after the PaLM-E model receives a demand in natural language and responds to the demand, when it needs to recognize an image for the next step of response, it still needs to process the acquired image in the form of an image (processed by the visual model ViT of the PaLM-E model). The specific process is as follows:

[0007] 1) Split the input image into fixed-size fragments (e.g. 16x16) and project each fragment into a vector as the input sequence of the Transformer;

[0008] 2) Add a special category tag (CLS) to the input sequence to indicate the category information of the image; at the same time, add a position code to each fragment to indicate the position information of the fragment in the image;

[0009] 3) The input sequence is fed into the Transformer encoder, and multi-layer self-attention and feed-forward network calculations are performed to obtain the output vector of each fragment.

[0010] 4) Take out the output vector corresponding to the category label, pass it through a linear layer and a Softmax layer, and get the category prediction probability of the image.

[0011] If more modal inputs need to be processed, the corresponding large-scale multimodal model needs to combine more large-scale models of different modalities, requires a larger data set for training, and requires greater computing power for the calculation of the large-scale multimodal model.

[0012] On the other hand, in the prior art, the collection of sensory data and the processing of the input sensory data using artificial intelligence models usually occur in a point-by-point manner, or in a repeated point-by-point manner at different locations. For example, identity confirmation or recognition usually uses a point-by-point manner, that is, recognition is performed only at one location, or repeated recognition is performed at multiple locations.

[0013] When confirming or identifying a person's identity, the method is: user password verification or biometric recognition is performed at the location where identification is required; the shortcomings are: on the one hand, there are major security risks. When the password is stolen or the face is imitated, there is a risk of identity impersonation regardless of whether the identification is performed at one location or repeated identification is performed at multiple locations; on the other hand, users are required to actively output information, and the user experience is poor.

[0014] When confirming or identifying the identity of a vehicle, the method is: feature recognition; the disadvantages are: the recognition result is easily broken, the data source is not completely reliable, the recognition technology is required to be high, but the recognition accuracy is low.

[0015] In addition, the single-point recognition method requires high-speed recognition operations at each point, which has high hardware requirements and will reduce execution efficiency as the amount of data increases.

[0016] Similarly, other application scenarios also have the shortcomings of single-point identification when they are applied in a point-by-point manner.

[0017] It can be seen that large-scale multimodal models are still in the laboratory stage. When they are applied in production, problems such as efficiency, cost, reliability, and safety need to be solved. Summary of the invention

[0018] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a self-processing method based on perceiving the real world, which can simultaneously collect multimodal perception data and use a dedicated large language model for processing, so as to complete the self-processing of the execution word vector that describes the multimodal perception data input in real time; the symbolization also includes simplifying and transcoding the multimodal perception data and the corresponding processing result data through encoding, obtaining the training encoding corpus and the execution encoding corpus in the symbol combination format of simplifying and transcoding the multidimensional matrix, and combining them with the training word vector and the execution word vector to perform model optimization training and self-processing, thereby achieving the effects of high efficiency, low cost, high reliability, high security, etc. of optimization training and self-processing, which can be used for perception of the real world and accurately perform automatic response and processing.

[0019] The technical solution of the present invention is as follows:

[0020] A self-processing method based on perceiving the real world comprises the following steps:

[0021] 1) Optimizing the training step: using various types of data acquisition devices deployed in the working area to collect various types of corresponding types of perception data, the collected perception data of various types cover the objects within the monitoring range of the corresponding type of data acquisition device; obtaining the processing result data corresponding to the collected perception data; symbolizing the various types of perception data collected, including but not limited to the processing result data corresponding to the collected perception data, to form a training word vector with natural semantics, the training word vector describes the multimodal perception data and the processing result data corresponding to the collected perception data, and the training word vector is used as training data, input into a general large language model, and the general large language model is optimized and trained to obtain a special large language model;

[0022] 2) Self-processing step: Utilize various types of data acquisition devices deployed in the working area to collect various corresponding types of perception data, symbolize the collected perception data of each type, and form execution word vectors with natural semantics. The execution word vectors describe the multimodal perception data, and use the execution word vectors as real-time data to input into a dedicated large language model to generate self-processing result data with natural semantics, and perform corresponding self-processing operations based on the self-processing result data.

[0023] Preferably, in step 1), taking functional scenarios as units, various types of perception data collected in different functional scenarios and training word vectors corresponding to the processing result data corresponding to the collected perception data are used to input into a general large language model for optimization training to generate a special large language model; the special large language model generates the processing result data based on the execution word vector; wherein, there is no need to predefine the functional scenarios.

[0024] Preferably, step 1) specifically comprises: utilizing various types of data acquisition devices deployed in the working area to continuously acquire various corresponding types of perception data, wherein the continuously acquired perception data of various types include perception data in chronological order acquired from objects in the working area covered by the monitoring range of the corresponding type of data acquisition device; symbolizing the continuously acquired perception data of various types and the processing result data corresponding to the acquired perception data, converting them into training word vectors with natural semantics, and inputting the training word vectors into a general large language model for optimization training to generate a dedicated large language model.

[0025] Preferably, a three-dimensional global model is obtained by three-dimensional modeling, wherein the three-dimensional global model includes a three-dimensional object model representing the object;

[0026] In step 1), various types of corresponding virtual acquisition devices are deployed in the three-dimensional global model; the virtual acquisition devices are used to perform in-model acquisition on the three-dimensional object model to obtain in-model perception data; the in-model training word vectors corresponding to the in-model perception data are input into the dedicated large language model to generate in-model self-processing result data, and an automated script is generated corresponding to the in-model self-processing result data; the in-model self-processing result data is an in-model processing solution for the behavior, attribute, state or relationship between objects of the three-dimensional object model at the moment when the in-model perception data is collected; the in-model self-processing result data is used as the in-model processing result data corresponding to the in-model perception data; the three-dimensional object model corresponding to the in-model processing result data is controlled by an automated script or manually controlled to execute the in-model processing solution in the three-dimensional global model, thereby optimizing and training the dedicated large language model.

[0027] Preferably, the symbolization described in step 1) and step 2) further includes, by encoding, converting the various types of sensory data described in step 1) including but not limited to the collected sensory data, the processing result data corresponding to the collected sensory data, and the various types of sensory data collected in step 2) to obtain respectively the training coded corpus and the execution coded corpus in the symbol combination format for simplifying transcoding of the multidimensional matrix, specifically:

[0028] Different types of perception data belonging to different objects collected by different types of data collection devices and the processing result data corresponding to the collected perception data are simplified and transcoded based on transcoding rules to obtain matching symbol simple codes; a perception multidimensional matrix is ​​constructed for the symbol simple codes corresponding to all current perception data collected by the data collection device; the symbol combination format includes but is not limited to one or more combinations of texts, characters, super-large base sequences, known symbols of non-natural languages, and custom symbols of various natural languages;

[0029] In step 1), the training coded corpus is combined with the training word vector as training data, and input into the general large language model to optimize the training of the general large language model; in step 2), the execution coded corpus is combined with the execution word vector as real-time data, and input into the dedicated large language model to generate self-processing result data with natural semantics.

[0030] Preferably, if in step 1), new training word vectors are obtained using perception data of different working areas, different objects, and different modalities, and the dedicated large language model is optimized and trained, the dedicated large language model is optimized and trained to enable the dedicated large language model to increase adaptation of perception data of different objects and different modalities.

[0031] Preferably, a peer-to-peer computing system is set up, the peer-to-peer computing system includes multiple node devices, and there is no primary-secondary relationship between all node devices; the node devices of the peer-to-peer computing system are used to collect various types of perception data related to the target, and the target includes an object already existing in the working area and an object entering the working area;

[0032] The node device deploys a vectorizer for symbolizing the perception data; for a certain node device, the vectorizer is used to symbolize the various types of perception data collected and the processing result data corresponding to the collected perception data to form a training word vector or an execution word vector, and the training word vector or the execution word vector is processed as the original data to obtain the result data, and the result data is propagated to other node devices; other node devices that receive the result data use the result data as one of the collected original data, and influence the result data of other node devices through the result data;

[0033] Based on this, multiple node devices in the peer computing system perform collaborative computing. When the result data of the collaborative computing can determine the target perception result, the target corresponding to the multimodal perception data is perceived to obtain the target perception result, wherein the target perception result is a comprehensive state of one or more combinations of the target's behavior, attributes, state, or the relationship between objects, and is represented in the result data.

[0034] Preferably, the node device is connected to and controls the corresponding execution device;

[0035] For a certain node device, a portion of the self-processing result data corresponding to the connected execution device is processed as original data to obtain result data, and the result data is transmitted to other node devices; other node devices that receive the result data use the result data as one of the collected original data, and influence the result data of other node devices through the result data;

[0036] The node devices in the peer computing system perform collaborative computing as the original data is acquired and the result data is calculated; based on the collaborative computing, the node devices respond to the self-processing result data until the result data calculated by one or more node devices matches the request content corresponding to the self-processing result data;

[0037] The self-processing result data represented by the result data is symbolized by a vectorizer deployed on the node device, and converted into a control word vector corresponding to the execution device, and the execution device is controlled by the control word vector; or, the output control word vector is used as a request, the result data obtained by calculation includes the corresponding control word vector, and the execution device is directly controlled by the control word vector.

[0038] Preferably, in step 1), various types of perception data related to the target are collected by using node devices of the peer computing system; for a certain node device, the training word vector formed by the collected perception data of various types and the processing result data corresponding to the collected perception data is obtained;

[0039] When using training word vectors for optimization training, the node device receives the assigned optimization training tasks and training word vectors, and performs corresponding optimization training; processes the training results as raw data to obtain result data, and propagates the result data to other node devices; other node devices that receive the result data use the result data as one of the collected raw data, and influence the result data of other node devices through the result data;

[0040] The node devices in the peer computing system perform collaborative computing as the original data is acquired and the result data is calculated; based on the collaborative computing, the training results are responded to until the result data calculated by one or more node devices meets the goal of optimizing the training.

[0041] Preferably, when optimizing and training the general large language model using the training word vector, the collected perception data of various types and the processing result data corresponding to the collected perception data are stored in the data storage module of the data collection device connected to the corresponding node device;

[0042] The perception data and the corresponding processing result data are called and output as requests to calculate the result data. After calculating the result data, the corresponding node device in the peer computing system drives the data storage module of the connected data acquisition device to establish a file transfer channel between the node devices of the peer computing system or to establish a file transfer channel in other network communication modes, and sends the perception data and the corresponding processing result data to a specific location.

[0043] Preferably, in step 1), various types of perception data related to the target are collected by using node devices of the peer computing system; for a certain node device, the training word vector formed by the collected perception data of various types and the processing result data corresponding to the collected perception data is obtained;

[0044] The training word vector is used as training data and input into a neural network for generating a general large language model, and the general large language model is trained and generated; when performing model training, the node device receives the assigned model training task and the training word vector and performs corresponding model training; the training result is used as raw data for processing to obtain result data, and the result data is propagated to other node devices; other node devices that receive the result data use the result data as one of the collected raw data, and influence the result data of other node devices through the result data;

[0045] The node devices in the peer computing system perform collaborative computing as the original data is acquired and the result data is calculated; based on the collaborative computing, they respond to the training results until the result data calculated by one or more node devices meets the goal of model training.

[0046] Preferably, in step 2), various types of perception data related to the target are collected by using node devices of the peer computing system; for a certain node device, an execution word vector formed by the collected various types of perception data is obtained;

[0047] When using a dedicated large language model to generate self-processing result data, the node device receives the assigned generation task and generates the corresponding part of the self-processing result data; the generated result is processed as the original data to obtain the result data, and the result data is transmitted to other node devices; other node devices that receive the result data use the result data as one of the collected original data, and influence the result data of other node devices through the result data;

[0048] The node devices in the peer computing system perform collaborative computing as the original data is acquired and the result data is calculated; based on the collaborative computing, the generated results are responded to until the result data calculated by one or more node devices matches the self-processing result data.

[0049] Preferably, the node device deploys an encoder for generating coded corpus for training and coded corpus for execution, and a decoder matching the encoder;

[0050] Inputting the various types of sensory data collected in step 1), including but not limited to, the processing result data corresponding to the collected sensory data, or the various types of sensory data collected in step 2), into an encoder to generate a training coded corpus or an execution coded corpus in a symbol combination format;

[0051] The training coded corpus or execution coded corpus in the symbol combination form of the simple code value + the correction code is used as one of the raw data of the calculation result data, and the result data obtained by the calculation is transmitted to the subsequent node device;

[0052] The subsequent node device receives the result data, uses a decoder to restore the simple code value + correction code as the training coded corpus or the execution coded corpus from the result data, and then translates the simple code value back to the original code parameter string, and disassembles the original code parameter string into a number of original code parameters corresponding to the current node device; translates the correction code back to the correction value string, and disassembles the correction value string into a number of correction values ​​corresponding to the current node device; adds the correction value to the first value of the level section corresponding to the original code parameter, and restores the level of the type of perception data sent by the current node device;

[0053] Based on this, when there is a data restoration request, the result data is combined with the decoder to restore the various types of perception data collected in step 1), including but not limited to the processing result data corresponding to the collected perception data, and the various types of perception data collected in step 2).

[0054] Preferably, in step 1), the sensory data of various types collected corresponding to the coded corpus for training, including but not limited to the sensory data of various types collected, and the processing result data corresponding to the sensory data collected are not stored;

[0055] In step 2), the perception data of various types corresponding to the encoding corpus for execution are not stored.

[0056] Preferably, when the result data calculated by one or more node devices in the peer computing system matches the request content corresponding to the self-processing result data, the result corresponding to the request content is represented in the result data output by one or more node devices according to preset conditions or the output of a pre-deployed program or a data processing model deployed on the node device; if it is determined that the current node device needs to respond to the request command, the current node device will send a control instruction to the execution device connected to the current node device according to the calculated result data; when all the execution devices connected to all the node devices that need to respond have completed the response action, the self-processing operation is completed.

[0057] The beneficial effects of the present invention are as follows:

[0058] The self-processing method based on perceiving the real world described in the present invention, in the steps of optimizing the training of a general large language model and using a special large language model to respond to input to realize self-processing, the various types of perceptual data collected in the two steps, the processing result data corresponding to the collected perceptual data obtained in the optimization training step, that is, the multimodal perceptual data and the corresponding processing result data are symbolized to form a training word vector with natural semantics and an execution word vector, the training word vector describes the multimodal perceptual data and the processing result data corresponding to the collected perceptual data, and the execution word vector describes the multimodal perceptual data; further, the perceptual data of multiple different modes are transformed into word vectors with natural semantics, and the amount of information contained in the word vector obtained by symbolization is approximately equal to the sum of the amount of information contained in the perceptual data of multiple different modes and the corresponding processing result data. Based on this, the present invention can word vectorize multimodal perceptual data and is suitable for the optimization training of a general large language model, thereby reducing the complexity of the optimization training of the special large language model, improving the training efficiency and response efficiency, and reducing the training cost and implementation cost.

[0059] In the present invention, the symbolization also includes simplifying and transcoding the multimodal perceptual data and the corresponding processing result data through encoding, obtaining the training coded corpus and the execution coded corpus in the symbol combination format of simplifying and transcoding the multidimensional matrix, and combining with the training word vector and the execution word vector to perform model optimization training and self-processing. After the technical solution is implemented, the type of the data acquisition device is known, that is, the type of the perceptual data is known, that is, the modality of the perceptual data is known, and after determining the processing result data corresponding to the collected perceptual data, the inherent laws between each type of perceptual data, multimodal perceptual data, and the corresponding processing result data can be covered. Therefore, the training coded corpus and the execution coded corpus obtained by simplified transcoding have specific inherent laws, and the context logic is clear, while ensuring the training efficiency, the accuracy of the output of the dedicated large language model is improved. The training coded corpus and the execution coded corpus have a certain correspondence to the perceptual data they represent, and the training coded corpus and the execution coded corpus do not necessarily have to be related to the natural language vocabulary. For large language models, the encoded corpus used for training and the encoded corpus used for execution can be regarded as equivalent to the "language" used between sensors.

[0060] Based on this, when the present invention performs optimization training of the model, the multimodal perception data and the corresponding result processing data are converted into training word vectors with natural semantics, or training coded corpus in combination with symbol combination format, and then the general large language model is optimized and trained using the training word vectors or the training coded corpus; when the dedicated large language model is used to respond to the input execution word vectors with natural semantics, or the execution coded corpus in combination with symbol combination format to achieve self-processing, the processing of more complex multimodal perception data is achieved in a relatively simple way. That is, in the present invention, what is input into the dedicated large language model is not the perception data itself, and the dedicated large language model does not need to perform corresponding processing on perception data of different modalities. For example, when it is necessary to respond to the collected image, the dedicated large language model does not directly receive the image data, but receives the symbolized (including word vectorization, or combined with simplified transcoding) image data, and then uses the dedicated large language model to process the symbolized real-time data, generate self-processing result data, and then perform corresponding self-processing operations based on the self-processing result data.

[0061] After the deployment of the dedicated large language model is completed, the present invention continuously collects different types of perception data of all objects through the data collection device deployed in the working area, and after symbolization, the dedicated large language model is used to respond to the input real-time data to complete self-processing, thereby realizing the perception of the real world. Based on the technical solution of the present invention, the deficiency that the large-scale language model of the prior art and the real world cannot form a corresponding relationship can be perfectly avoided. Without the need to combine large-scale models of other modalities, it is still possible to realize the perception of the multi-modal real world (the perception recorded in the present invention is a different definition from the perception data collection. The perception means completing the perception of the target or object corresponding to the multi-modal perception data to obtain the target perception result).

[0062] At the same time, including but not limited to laws and regulations, implementation standards, specifications, and custom constraints are also used as perception data, and thus, as a type of training data. When other training data are real data for perceiving the real world, it is ensured that the behaviors and events corresponding to the training data input into the big model are carried out under the constraints of the laws, regulations, and scientific laws of the real world, thereby avoiding the provision of false and one-sided training data and preventing the accuracy of the big model's self-processing and functional realization from being affected.

[0063] The present invention optimizes and trains the general large language model based on functional scenarios. It can use a transformer model for unsupervised learning, and can complete training for different technical fields without pre-defining functional scenarios. Based on a large-scale data set, a dedicated large language model under certain conditions can be obtained.

[0064] The present invention can also utilize a three-dimensional model to optimize and train a dedicated large language model in the three-dimensional model. The present invention can build a three-dimensional global model that simulates the real world, or use the real world as a prototype to build a three-dimensional global model that is basically the same as the real world. The three-dimensional global model contains three-dimensional object models of all corresponding virtual objects or real objects, and they move and interact with each other according to certain rules. Then, by collecting different types of perception data from the three-dimensional object model and providing processing result data corresponding to the collected perception data, the dedicated large language model is optimized and trained. By optimizing and training the dedicated large language model through the three-dimensional global model, with the assistance of an automated script, the input of manpower can be reduced, and time utilization can be maximized, thereby improving the efficiency of optimizing and training the dedicated large language model and reducing costs.

[0065] The present invention utilizes a peer-to-peer computing system to calculate and transmit result data through collaborative computing (the result data calculated and transmitted does not include training data and real-time data such as word vectors for word vectorization, word vectors for execution, coded corpus for training in symbol combination format, coded corpus for execution, etc., but is the calculation result obtained by calculating the training data and real-time data based on the corresponding data processing mode, that is, when transmitting the result data, the training data and real-time data are not directly transmitted), and then optimize the training of the general large language model and respond to the input. In the described peer-to-peer computing system, there is no master-slave relationship between all node devices, and there is no fixed connection path between node devices. The node device only receives the calculation results of other node devices (that is, the result data calculated by other node devices), and sends out the result data of its own calculation. The optimization training and / or the response to the input do not rely on a single node device for processing and control, but are completed by collaborative computing and the transmission of result data through multiple node devices in the peer-to-peer computing system.

[0066] Furthermore, the optimization training and response to input of the present invention do not rely on point-to-point occurrence, and the computing functions are distributed throughout the entire peer-to-peer computing system, which not only reduces the hardware and software requirements of single-point computing, has high execution efficiency, and greatly improves the ability to resist attacks; but also presents relatively symmetrical information between node devices, and is immune to the problem of illegal data tampering. Even if a single node device is physically cracked and the data it sends is tampered with, because the calculation of the peer-to-peer computing system is a complex calculation with ultra-high redundancy and ultra-multi-dimensional verification, the tampering of the data sent by a single node device does not affect the calculation results of the peer-to-peer computing system, and the fault and the tampered node device can be quickly located to ensure the credibility of the calculation results of the peer-to-peer computing system.

[0067] In the present invention, especially in small-scale scenarios and short-time application scenarios, simplified transcoding is used to obtain coded corpus for training and coded corpus for execution, and based on a peer computing system, the information volume of the original data is integrated into the result data through collaborative calculation. When optimizing training and self-processing, the collected original data (i.e., step 1) and step 2) collected perception data and corresponding processing result data) do not need to be stored and can be restored, thereby saving storage costs and being immune to attacks on the model by providing false or defective training data. DETAILED DESCRIPTION

[0068] The present invention is further described in detail below with reference to the embodiments.

[0069] In order to solve the shortcomings of the prior art such as low efficiency, high cost, poor reliability and low security, the present invention provides a self-processing method based on perceiving the real world, which symbolizes (including word vectorization and simplified transcoding) the training samples used as optimization training, the different types of perception data used as input, and the corresponding processing result data, that is, the perception data of multiple different modalities collected separately, and the corresponding processing result data, optimizes the training to obtain a special large language model, and uses the special large language model to process the symbolized (including word vectorization and simplified transcoding) perception data of multiple different modalities, so as to generate self-processing result data corresponding to the perception data of multiple different modalities.

[0070] The present invention provides a self-processing method based on perceiving the real world, comprising the following steps:

[0071] 1) Optimize the training step: use various types of data acquisition devices deployed in the working area to collect (preferably, continuously collect) various corresponding types of perception data, and the collected perception data of various types covers the objects within the monitoring range of the corresponding type of data acquisition device; obtain the processing result data corresponding to the collected perception data (that is, the different types of perception data collected for the objects in the working area, the corresponding data output, such as the control instructions that need to be issued to the execution device, the interactive information that needs to be output for specific personnel, and other processing solutions, and the processing result data corresponding to the collected perception data include but are not limited to the control instructions, interactive information, etc.);

[0072] The various types of sensory data collected, including but not limited to the above (since the present invention is implemented in the real world, and the real world operates based on various legal and human constraints, in specific implementation, the sensory data also includes but is not limited to laws and regulations, execution standards, specifications, and custom constraints, and thus, the self-processing result data output by the dedicated large language model can comply with the operating rules of the real world) and the processing result data corresponding to the collected sensory data are symbolized, such as word vectorization, to form training word vectors with natural semantics, wherein the training word vectors describe multimodal sensory data and the processing result data corresponding to the collected sensory data, and the training word vectors are used as training data, input into the general large language model, and the general large language model is optimized and trained to obtain the dedicated large language model;

[0073] 2) Self-processing step: Utilize various types of data collection devices deployed in the work area to collect various corresponding types of perception data, symbolize the collected perception data of various types, such as word vectorization, to form execution word vectors with natural semantics, the execution word vectors describe the real-time data of multimodal perception data; use the execution word vectors as real-time data, input them into a dedicated large language model, generate self-processing result data with natural semantics, and perform corresponding self-processing operations based on the self-processing result data. The self-processing result data generated by the dedicated large language model includes but is not limited to interaction data and / or task strategies, and correspondingly, the self-processing operations include but are not limited to outputting interaction data for human-computer interaction or machine-machine interaction and / or controlling objects corresponding to task strategy control.

[0074] Specifically, when the self-processing result data is interactive data, the interactive device is controlled to output the interactive data through the matching task strategy; when the self-processing result data is a task strategy, the corresponding object is controlled by the task strategy to perform the corresponding operation. Wherein, the control instructions and instruction sequence contained in the task strategy are used to control the corresponding object to perform the corresponding operation based on the task strategy; wherein, the corresponding object controlled based on the task strategy is at least one object whose sensed data is collected and controlled, or at least one object is set (such as at least one controlled object set on an object whose sensed data is collected but not controlled), and various working parameters of the object are obtained as sensed data; the object responds to the various working parameters contained in the processing scheme to control the object. The interactive data includes but is not limited to voice interactive data, image interactive data, and text interactive data, and data interaction is performed through interactive devices of corresponding types.

[0075] The present invention is based on perception data, and the perception data collected from the real world (the perception of the real world described in the present invention is defined differently from the collection of perception data, and the perception represents the completion of the perception of the target or object corresponding to the multimodal perception data to obtain the target perception result) are all objective parameters, and different types of perception data and their mutual relationships are subject to the laws of the real world; at the same time, the training word vectors with natural semantics can be accurately understood by the general large language model, and then, the collected perception data and the corresponding result processing data are symbolized into training word vectors with natural semantics as input. When there is a sufficient amount of data to optimize the general large language model for training, the special large language model generated by the optimization training is used to predict or calculate the output, which can achieve more accurate adaptability.

[0076] The symbolization described in the present invention, that is, through the word vectorization method, the various types of perception data collected, including but not limited to the processing result data corresponding to the collected perception data, are symbolized to form training word vectors with natural semantics. This occurs before inputting into the general large language model, and is different from the vectorization or vectorization performed after the training samples are input into the neural network. Due to the different timing, location, and role played in the technical solution, they should be clearly distinguished.

[0077] After the deployment of the dedicated large language model, the present invention continuously collects different types of perception data of all objects through the data collection device deployed in the working area, and after symbolization, the dedicated large language model is used to respond to the continuously input real-time data to complete self-processing, thereby realizing the perception of the real world. Based on the technical solution of the present invention, the deficiency that the large-scale language model of the prior art and the real world cannot form a corresponding relationship can be perfectly avoided, and the perception of the multi-modal real world can still be realized without the need to combine large-scale models of other modalities.

[0078] In step 1) described in the present invention, the training of a dedicated large language model is performed in units of functional scenarios; wherein the functional scenarios are application scenarios for realizing certain functions, such as robots automatically moving vehicles, automatic garbage processing, automatic fire fighting, etc., which can realize a specific event. Since the present invention uses symbolized training data to optimize the general large language model, when the training data includes perception data and corresponding processing result data, there is no need to predefine the functional scenarios, that is, unsupervised learning can be used to achieve optimization training. In specific implementation, the occurrence and end of a specific event are used as the starting and ending points of the scenario, and the training word vectors corresponding to the various types of perception data collected in different functional scenarios and the processing result data corresponding to the collected perception data are used to input into the general large language model for optimization training to generate a dedicated large language model. Furthermore, in the step 2), the dedicated large language model generates self-processing result data based on the execution word vector as real-time data; that is, the dedicated large language model has the versatility dedicated to various functional scenarios (different from the general large language model, which is indiscriminately applicable to various functional scenarios, but the general large language model cannot accurately calculate and output accurate self-processing result data for the execution word vector without optimization training). After the optimization training is completed, if necessary, if the self-processing result data output for the uninvolved functional scenario cannot meet the demand, the uninvolved functional scenario can be optimized and trained to achieve the increased adaptation of the functional scenario of the dedicated large language model. Specifically, if in step 1), new training word vectors are obtained using perception data of different working areas, different objects, and different modalities, and the dedicated large language model is optimized and trained, then the dedicated large language model is optimized and trained to achieve the increased adaptation of the dedicated large language model to the perception data of different objects and different modalities, which is the process of fine-tuning the model.

[0079] In order to take into account the convenience and efficiency of optimization training, in step 1), when there are multiple independent functional scenarios with the same functions, for example, there are N functional scenarios of automatic handling and parking, all of them are used for optimization training. First, according to the selection rules, some of the multiple (such as N) functional scenarios are automatically selected, and then the selected several (such as M of the N functional scenarios) functional scenarios are input into the general large language model for optimization training. Then, more suitable scenarios can be screened out for optimization training to achieve the training expectations. Among them, the multiple independent functional scenarios with the same functions are not a functional scenario copied into multiple ones, but a plurality of independent, similar functional scenarios with the same functions are set. For example, there are multiple functional scenarios where robots automatically move vehicles. The multiple functional scenarios have the same function, that is, "there are vehicles parked in a specific area, and the robot automatically moves the vehicle to other specific areas", and the multiple functional scenarios are independent and not completely the same.

[0080] In order to facilitate manual or automatic identification and sorting of multiple functional scenarios to improve the efficiency of optimization training, in specific implementation, further, after automatically selecting several functional scenarios, the training word vectors are output first, or, when the functional scenarios are used to optimize the training of the general large language model, the collected perception data of various types and the processing result data corresponding to the collected perception data are stored, and then called and output (such as using the peer computing system provided by the present invention to call the original data stored in a specific location in a specific path), and then manually or automatically identify the output training word vectors or perception data and the corresponding processing result data to prioritize the selected functional scenarios; and then, the functional scenarios with the best training effect are given priority for optimization training, so that a dedicated large language model that is available to a certain extent can be obtained within the shortest possible training time.

[0081] Furthermore, in order to make the degree of optimization training controllable, in specific implementation, the input start and end points of the training word vector (or the training coded corpus described in the present invention, together as training data, the same below) for the optimization training of the functional scenario can also be set manually, wherein the input start and end points of the training word vector are the state of the training word vector starting input and the state of the training word vector ending input, that is, for example, the training word vector is set to start input at which time and end input at which time. Furthermore, when the output accuracy of the dedicated large language model is better than the threshold, optimization training can be performed to ensure output accuracy.

[0082] In the present invention, step 1) is more specifically to continuously collect various types of corresponding types of perception data using various types of data acquisition devices deployed in the working area, and the continuously collected perception data of various types include the perception data collected from the objects in the working area covered by the monitoring range of the corresponding type of data acquisition device with a time sequence; the continuously collected perception data of various types and the processing result data corresponding to the collected perception data are symbolized, converted into training word vectors with natural semantics, and the training word vectors are input into the general large language model for optimization training to generate a special large language model. Among them, when symbolizing the continuously collected perception data and the processing result data corresponding to the collected perception data, it can be implemented as a single collection, a single symbolization, or multiple connection collections and collective symbolization, wherein the perception data and processing result data for collective symbolization can be selected in random quantity, fixed quantity cycle selection, or other set rules (such as selecting nodes with value changes, or selecting cycles with cyclic changes). The implementation methods of different quantity selections are all based on the selection criteria of improving symbolization and optimizing training efficiency of the corresponding type of perception data.

[0083] In the present invention, when the trained dedicated large language model is used to output the self-processing result data, it can theoretically cover all prominent conditions in the working area, including the target effect corresponding to the self-processing result data; but in order to provide a redundant guarantee scheme and ensure the accuracy and safety of the self-processing of the present invention, in step 2), the collected perception data is symbolized to obtain the execution word vector at the starting moment (or combined with the execution encoding corpus described in the present invention, together as real-time data, the same below), and based on the dedicated large language model, the corresponding complete task self-processing result data is obtained; following the execution of the previous self-processing operation, the perception data is collected and symbolized in real time, the execution word vector at the current moment is obtained, and the corresponding remaining task self-processing result data is generated based on the dedicated large language model. Wherein, the complete task self-processing result data includes the target interaction data and / or target task strategy corresponding to the moment when the execution word vector at the starting moment is collected, and the remaining task self-processing result data includes the verification interaction data and / or verification task strategy corresponding to the moment when the execution word vector at the current moment is collected each time.

[0084] When the verification interaction data and / or verification task strategy contained in the remaining task self-processing result data are different from or contradictory to the target interaction data and / or target task strategy contained in the complete task self-processing result data, the complete task self-processing result data is judged to be invalid, and then, the remaining task self-processing result data is used as the replacement self-processing result data, and the corresponding self-processing operation is performed based on the replacement self-processing result data. Based on this, the self-processing result data is automatically optimized or automatically adjusted. Among them, after the remaining task self-processing result data is generated, it does not affect the self-processing based on the complete task self-processing result data. Before the remaining task self-processing result data replaces the complete task self-processing result data, it is only used to verify whether the complete task self-processing result data is valid for the perception data at the current moment.

[0085] Furthermore, the self-processing result data is used as training data to optimize the dedicated large language model.

[0086] In order to improve the efficiency of optimization training of the dedicated large language model, the present invention also uses software to assist in optimization training. Specifically, a three-dimensional global model is obtained through three-dimensional modeling, and the three-dimensional global model includes three-dimensional object models representing objects (three-dimensional object models move and interact with each other according to certain rules); wherein the virtual area corresponding to the three-dimensional global model corresponds to the working area of ​​the real world, and the three-dimensional object model, as a virtual object, corresponds to the object of the real world.

[0087] Correspondingly, in step 1), various types of corresponding virtual acquisition devices (corresponding to data acquisition devices in the real world) are deployed in the three-dimensional global model; the virtual acquisition devices are used to perform in-model acquisition on the three-dimensional object model to obtain in-model perception data; the in-model training word vectors corresponding to the in-model perception data are input into the dedicated large language model to generate in-model self-processing result data, and an automation script is generated corresponding to the in-model self-processing result data;

[0088] The self-processing result data in the model is the model-in-processing solution for the behavior, attribute, state or relationship between objects of the three-dimensional object model at the moment when the model-in-perception data is collected; the self-processing result data in the model is used as the model-in-processing result data corresponding to the model-in-perception data;

[0089] The three-dimensional object model corresponding to the processing result data in the model is controlled by automated scripts or manually controlled to execute the model processing solution in the three-dimensional global model, thereby optimizing the training of the dedicated large language model.

[0090] Since the three-dimensional global model can be separated from the real world and accurately set based on expectations, compared with the real world, the training data and functional scenarios are completely controllable, and the interference of unexpected situations on the training process can be almost avoided, including the obstruction of buildings and trees that may exist in the real world, or the selection of the location of the data acquisition device, etc., which have to be compromised or avoided according to the actual situation in the real world; and the perception data and processing result data collected within the model are the same as the perception data and processing result data collected in the real world, are the same type of data, and play the same role; and it is capable of long-term, continuous, and massive data training, greatly shortening the time span of optimization training.

[0091] In the present invention, in order to ensure that the real world can be completely replicated when the three-dimensional global model is used for optimization training of a dedicated large language model, in specific implementation, a modeling area is demarcated corresponding to the working area; three-dimensional modeling is performed on the real objects in the modeling area to construct a three-dimensional global model; the three-dimensional global model includes a three-dimensional object model corresponding to the real object;

[0092] Correspondingly, in step 1), corresponding to various types of data acquisition devices in the working area, corresponding various types of virtual acquisition devices are deployed in the three-dimensional global model; the virtual acquisition devices are used to perform in-model acquisition on the three-dimensional object model to obtain the perception data within the model.

[0093] Furthermore, when implementing long-term, continuous, and massive data training, it can also ensure that the optimization training performed using the three-dimensional global model is nearly consistent with the training results performed in the real world.

[0094] In the present invention, the three-dimensional modeling, three-dimensional global model, and three-dimensional object model are three-dimensional models formed by image data through modeling, rendering, and other processes using a three-dimensional engine. When the three-dimensional global model is used for optimization training, the three-dimensional global model can be automatically operated through a script. As the three-dimensional global model runs, the model is collected, self-processing result data in the model is generated, and an automated script is generated. The optimization training process may not be shown throughout the process. When the implementation requirements require the optimization training process, especially the operation process of the three-dimensional global model, the three-dimensional global model can be visualized for users to view or check.

[0095] In this embodiment, the symbolization described in step 1) and step 2) further includes, by encoding, converting the various types of sensory data described in step 1) including but not limited to the collected sensory data, the processing result data corresponding to the collected sensory data, and the various types of sensory data collected in step 2) to obtain training coded corpus and execution coded corpus in a symbol combination format for simplifying transcoding of a multidimensional matrix, specifically:

[0096] The different types of perception data belonging to different objects collected by different types of data collection devices and the processing result data corresponding to the collected perception data are simplified and transcoded based on the transcoding rules to obtain matching symbol simple codes; the perception multidimensional matrix is ​​constructed for the symbol simple codes corresponding to all current perception data collected by the data collection device, and then the perception multidimensional matrix is ​​simplified and transcoded again to complete the encoding. The symbol combination format includes but is not limited to various natural language texts, characters, super large base sequences, known symbols of non-natural languages, and one or more combinations of custom symbols.

[0097] Furthermore, in step 1), the training encoded corpus is combined with the training word vector as training data, and input into the general large language model to optimize the general large language model;

[0098] In step 2), the execution coded corpus is combined with the execution word vector as real-time data and input into a dedicated large language model to generate self-processing result data with natural semantics.

[0099] In specific implementation, when the symbol combination format is a text of various natural languages, it is sufficient as long as there is no duplication or conflict between the coded corpus for training, the coded corpus for execution, the word vectors for training, and the word vectors for execution.

[0100] Correspondingly, in order to further effectively simplify the complicated original data and facilitate the transcoding of the multidimensional matrix, in this embodiment, all symbol simple codes and the perception multidimensional matrix they constitute are exhaustively enumerated, and a certain simple code value is assigned to each perception multidimensional matrix; an M-bit N-ary symbol sequence is constructed, and each bit of the N-ary symbol sequence is assigned one of K different symbols; each specific value of the M-bit N-ary sequence is matched with each simple code value one by one; wherein the value of M×N is greater than the number of simple code values. In the specific implementation, the setting position, time, type and quantity of the perception data of the data acquisition device can be specifically defined, and then it is very simple to exhaustively enumerate all symbol simple codes and the perception multidimensional matrix they constitute, and the amount of data is not large. For example, if there are 100 million possibilities, 10,000 different symbols can be used to form a ten thousand sequence, that is, a 2-bit ten thousand sequence, such as X1X0, where X1 and X0 can be assigned one of 10,000 different symbols, and then, X1X0 has a total of 100 million values, corresponding to one of the 100 million possibilities. If a larger amount of information is needed during implementation, the number of digits in the ten-thousandth sequence can be increased according to the implementation requirements, such as X3X2X1X0, with a total of 10 16 Each value corresponds to one of 100 trillion possibilities.

[0101] Furthermore, in order to improve the efficiency of simplified transcoding, in this embodiment, the original data such as the perception data that needs to be simplified transcoded and the corresponding result processing data can be divided into multiple data blocks, and simplified transcoding is performed separately to obtain matching symbol codes, and then multiple data block corpora are obtained; each data block corpus is then used as the original data and simplified transcoded again to achieve nested transcoding, and finally the encoded corpus for training and the encoded corpus for execution are obtained.

[0102] In this embodiment, the transcoding rule is as follows: different types of perception data belonging to different objects collected by different types of data collection devices and the processing result data corresponding to the collected perception data are predefined into several levels, and all levels are divided into several level sections; a corresponding level section identifier is assigned to each level section, and a predefined type identifier is assigned to each type of perception data, and the type identifier + level section identifier constitutes the original code parameter; the difference between the level of a type of perception data and the first value of the level section to which it belongs is set as the correction value. In specific implementation, all original code parameter strings of different types of data collection devices are exhaustively enumerated, and a certain simple code value is assigned to each original code parameter string; all correction value strings of different types of data collection devices are exhaustively enumerated, and a certain correction code is assigned to each correction value string.

[0103] Arrange multiple groups of original code parameters that need to be simultaneously input into a general large language model or a special large language model to form an original code parameter string, arrange the correction values ​​to form a correction value string, convert the original code parameter string into a matching simple code value, convert the correction value string into a matching correction code, and use the simple code value + correction code as a coded corpus for training and a coded corpus for execution.

[0104] In order to achieve the purpose of simplifying the transmission content, the word length of the simplified code value is smaller than the word length of the original code parameter string, and the word length of the correction code is smaller than the word length of the correction value string. In the present invention, the number of bits of the simplified code value and the correction code is determined according to the number of exhaustively enumerated original code parameter strings and correction value strings, so that the number of original code parameter strings and correction value strings matched by the simplified code value and the correction code is not less than the number of exhaustively enumerated original code parameter strings and correction value strings.

[0105] As an implementation method that is easy to implement, in the present invention, if the number of exhaustive original code parameter strings and correction value strings is less than Q, where Q is the set number of symbols; then the simplified code value and the correction code are both represented by a one-bit symbol, and each original code parameter string and correction value string matches a one-bit symbol respectively. A one-bit symbol includes an i-bit j-base number combination, representing Q=i×j possible characters. Each character can be used to represent an original code parameter string or a correction value string, and an original code parameter string + correction value code represents a combination of a determined number of multiple sets of parameters transmitted in parallel each time.

[0106] Furthermore, if the number of exhaustively enumerated original code parameter strings and correction value strings is greater than the set number of symbols, each original code parameter string and correction value string matches at least two symbols, and multiple symbols are arranged to form a symbol string. Specifically, the symbol string includes a number of bit symbols, and the number of original code parameter strings and correction value strings matched by the symbol string is P = Q n , where n is the number of bits of the symbol, n=1, 2, 3, ... That is, in order to ensure that the number of possible characters included in the symbol string is greater than the number of exhaustive original code parameter strings and correction value strings, during implementation, the number of bits of the symbol string is determined according to the number of exhaustive original code parameter strings and correction value strings, so that the number of original code parameter strings and correction value strings matched by the symbol string is not less than the number of exhaustive original code parameter strings and correction value strings.

[0107] In the above embodiment, the symbols are used only as an embodiment for convenient explanation of the present invention. The simple code value and the correction code described in the present invention may also be other binary codes with a preset number of bits (such as 8 bits for the symbol, 1 byte), and 2 bits are formed according to the number of bits of the binary code. n Number of characters.

[0108] For example, the performance status of three devices a, b, and c is sent as an example. The performance status of devices a, b, and c are collected. The parameters of devices a, b, and c have 30 levels from 0 to 29. Exhaustive original code parameter strings can obtain 27,000 combinations. The data of the performance status of the three devices obtained is 9 characters, such as a15b27c18. In computer storage and transmission, these 9 characters need to occupy 9 8-bit binary codes, that is, a total of 9 bytes (72 bits) of information need to be transmitted.

[0109] In this embodiment, the parameters of each device are graded, such as being divided into six sections: a0-4, a5-9, a10-14, a15-19, a20-24, and a24-29. The same is true for devices b and c. A parameter section identifier is generated according to which level section the parameter level to be sent falls into. In this way, there are 216 combinations of parameter sections for the three devices, which can be represented by only one symbol. A parameter correction table is predefined in the current node device. The correction value represents the value of the parameter level floating on the level section to which it belongs, that is, the difference with the first value of the level section to which it belongs. For example, the correction values ​​of a15b27c18 are 0, 3, and 3 respectively, and the correction value string is 033. The parameter correction table is an exhaustive combination of correction value strings. In this embodiment, there are 125 correction value strings, which can also be represented by one symbol. It can be seen that the present invention can use two bytes to transmit 9 bytes of information. For example, the character corresponding to the combination a15b27c18 in the transcoding mapping table is "p", and the character corresponding to it in the parameter correction table is "k". Take "pk" as the original data and participate in peer calculation. The result data obtained by calculation is transmitted to the subsequent node device. The subsequent node device presets a transcoding mapping table and parameter correction table or decoder with the content completely consistent with the current node device. It is queried in the transcoding mapping table or determined through the decoder that "p" corresponds to the level segments of a15-19, b24-29, and c15-19. It is queried in the parameter correction table or determined through the decoder that "k" corresponds to 0, 3, and 3. The correction value is added to the first value of the level segment to restore a15b27c18.

[0110] Through the above process, 9 bytes of information can be transmitted using 2 bytes, thereby achieving the purpose of the present invention.

[0111] The symbolization described in the present invention can also be combined with multi-dimensional matrix coding, then the above example can be evolved into: the xyz coordinates of object A observed by camera 1 can be represented by abc (equivalent to recording data of the object in only three dimensions, and the same applies to multi-dimensional data), and then after simplified transcoding, the data of A in the multi-dimensional matrix is ​​pk. In the multi-dimensional matrix output by the node device, the observation results of camera 1 are A pk B mw, and the observation results of camera 2 are C ab D ef. The data of other sensors are similar, and finally a multi-dimensional matrix is ​​formed.

[0112] Based on the simplified transcoding, the result data obtained by calculation by the node device of the peer computing system provided by the present invention can be represented in a multidimensional matrix as the data of the multidimensional matrix, the state value of the multidimensional matrix data or the result of simplified transcoding of the multidimensional nested matrix data according to implementation requirements.

[0113] The simplified transcoding is one implementation method of symbolization. During specific implementation, a suitable encoding method can be selected for symbolization according to the implementation situation.

[0114] In the present invention, after the technical solution is implemented, the type of data acquisition device can be known, that is, the type of perception data can be known, that is, the modality of the perception data can be known, and after determining the processing result data corresponding to the collected perception data, the inherent laws between each type of perception data, multi-modal perception data, and corresponding processing result data can be included. Therefore, the training coded corpus and the execution coded corpus obtained by simplified transcoding have specific inherent laws, and the context logic is clear, while ensuring the training efficiency, the accuracy of the output of the dedicated large language model is improved. The simplified transcoding can be carried out by simple coding or complex multi-dimensional coding. Under the constraints of the rules of the real world, no matter which symbol is used for expression, it can cover specific inherent laws, and the context logic is clear. Although in almost all cases, it is not suitable for humans to read in the habit of natural language, it is particularly suitable for machine learning. Among them, the training coded corpus and the execution coded corpus have a certain correspondence to the perception data they represent, and the training coded corpus and the execution coded corpus do not necessarily have to be related to the natural language vocabulary. For large language models, the encoded corpus used for training and the encoded corpus used for execution can be regarded as equivalent to the "language" used between sensors.

[0115] The simplified transcoding described in the present invention will encode, including but not limited to, the various types of perception data collected and the processing result data corresponding to the collected perception data, and convert to obtain training encoded corpus and execution encoded corpus in the symbol combination format for simplified transcoding of multi-dimensional matrices; and the simplified transcoding occurs before the input of the general large language model, and is different from the feature extraction performed after the training samples are input into the neural network due to the different timing, location, and role played in the technical solution, and therefore, should be clearly distinguished.

[0116] The existing machine learning or deep learning technology, the vectorization or vectorization performed in the neural network, such as feature extraction, can learn some abstract and high-level representations from the data to describe the structure and rules of the data; for example, in image recognition, deep learning can extract edges, textures, shapes, objects and other features from pixel-level inputs, and these features rely on numerical matrices or tensors to represent the data; for example, in natural language processing, deep learning usually uses word embeddings or sentence embeddings to represent text, and these embeddings are real number vectors.

[0117] It can be seen that although the features extracted by traditional algorithms or deep learning in the prior art can be regarded as a formal transformation of image data and text data, they do not clearly define and use symbols, but rely on numerical matrices, tensors or real vectors to represent data rather than symbols or logical expressions; and the numerical representation obtained by extracting features is usually distributed, continuous, and high-dimensional, rather than discrete, sparse, and low-dimensional, and therefore is very different from the traditional cognitive symbol system.

[0118] Therefore, vectorization or vectorization for deep learning, although it can help understand the intrinsic structure and meaning of data, does not explicitly use and manipulate symbols, but relies on numerical calculations and optimizations, which is completely different from the simplified transcoding described in the present invention.

[0119] In contrast, the simplified transcoding described in the present invention can be implemented as using clearly defined, symbolic types to represent the object at the moment when the perception data is collected, based on one or more combinations of the object's behavior, attributes, state or relationship between objects reflected in the perception data. That is, the simplified transcoding described in the present invention is different from representing data in the form of numerical matrices, tensors or real vectors.

[0120] The difference between the simplified transcoding described in the present invention and the feature extraction of the prior art is that the timing of the operation is different, and correspondingly, the roles played in the respective technical solutions are different. The feature extraction of the prior art, for example, the feature extraction of deep learning generally occurs in the middle layer or the last layer of the convolutional neural network (CNN). These layers can extract high-dimensional feature vectors from the input image or text for subsequent classification, regression, matching and other tasks. The features extracted by different layers have different levels of abstraction and semantic information. Generally speaking, the closer the features are to the output layer, the more abstract they are, and the closer the features are to the input layer, the more specific they are.

[0121] The simplified transcoding described in the present invention occurs before inputting the general large language model, that is, the simplified transcoding is not necessarily performed using the general large language model. Furthermore, there is no difference in the abstractness or specificity of the features caused by the different locations where the features are extracted in the above-mentioned prior art.

[0122] Regarding traditional algorithms, the main difference between deep learning feature extraction and traditional methods is that deep learning is a data-driven approach that uses neural networks to automatically learn feature representation and abstraction, while traditional methods are manually designed to extract features using some fixed operators or patterns. The disadvantage of deep learning is that it requires a large amount of data and computing power, and its interpretability is weak. The disadvantage of traditional algorithms is that they have poor generalization and adaptability, and need to be specially designed for different tasks.

[0123] It can be seen that, similarly, the extraction features of the traditional algorithm are different from the simplified transcoding described in the present invention, the timing of the simplified transcoding described in the present invention is different, and the roles they play in their respective technical solutions are different.

[0124] Based on this, when the present invention performs optimization training, the multimodal perception data is converted into symbolic form, and then the training data containing the symbolic form is used for optimization training; when the model is used to respond to the input real-time data containing the symbolic form to achieve self-processing, the processing of more complex multimodal perception data is achieved in a relatively simple manner. That is, in the present invention, the input to the dedicated large language model may not be the perception data itself, and the dedicated large language model does not need to perform corresponding processing on perception data of different modalities. For example, when it is necessary to respond to the collected image, the image data is not used as the training data during optimization training, but the simplified transcoded image data is used as part of the training data, and then the dedicated large language model is used to process the data in symbolic form to generate self-processing result data, and then the corresponding self-processing operation is performed based on the self-processing result data.

[0125] In specific implementation, the traditional single-point recognition method can be used to collect perception data of the targets in the working area (including objects already existing in the working area and objects entering the working area) at the set position, and identify based on the perception data. For example, based on the collected images, image recognition can be used to determine what objects are contained in the image, the behavior, attributes, status of the objects, or the relationship between the objects. For example, based on the collected radio frequency signals, the identity of the corresponding associated objects can be determined, and so on. The collaborative computing based on the peer computing system provided by the present invention can also be used to complete the perception of the target. The present invention sets up a peer computing system, which is based on collaborative computing, does not rely on single-point identification, and distributes all computing functions in the peer computing system, thereby reducing the software and hardware requirements of single-point computing, having high execution efficiency, and greatly improving the ability to resist attacks. The information between node devices is relatively symmetrical, and can be immune to the problem of illegal data tampering. Even if a single node device is physically cracked and the data it sends is tampered with, because the calculation of the peer computing system is a complex calculation with ultra-high redundancy and ultra-multi-dimensional verification, the tampering of the data sent by a single node device does not affect the calculation results of the peer computing system, and the fault and the tampered node device can be quickly located, thereby ensuring the credibility of the calculation results of the peer computing system. Furthermore, the contradiction between data sharing and information security between departments can be resolved.

[0126] The result data transmitted between node devices can be the processing result of information rather than the information itself, so the collected original data (i.e., perception data) does not need to be stored. The node device only receives the calculation results output by other node devices and sends out its own calculation results. The amount of information contained in a single calculation result is not enough to restore any event and target information. The calculation results on the entire peer-to-peer computing system, the elements of the multi-dimensional data matrix, and the correspondence between the physical space and facilities must be jointly calculated to obtain a definite result. Collaborative computing is less dependent on the information transmitted by a few node devices, and can fundamentally change the nature of traditional information technology that is sensitive to single-point security.

[0127] The peer-to-peer computing system includes multiple node devices, and there is no master-slave relationship between all node devices, forming a decentralized network and computing architecture. Different from the traditional information-based single-point convergence computing mode, there is no fixed, preset path relationship in the direction of data transmission between the node devices of the present invention. In the peer-to-peer computing system described in the present invention, for a certain node device, the collected original data is processed to obtain result data, and the result data is propagated to other node devices; other node devices that receive the result data use the result data as one of the collected original data, and the result data of other node devices is affected by the result data. For the convenience of expression, the aforementioned "a certain node device" is referred to as the "current node device" and the "other node devices" are referred to as "subsequent node devices". Then one of the effects is that the result data calculated by the subsequent node device is not completely determined by the original data collected by itself, but is jointly determined with the result data output by the current node device; wherein, the result data output by the current node device may change the data processing model and parameters used by the subsequent node device to calculate the result data, thereby affecting the result data of the subsequent node device. For example, if the result data output by the current node device is correlated with the original data collected by the subsequent node device, it is necessary to consider the impact of the result data output by the current node device on the accuracy of the result data of the subsequent node device; specifically, for the perception of a specific target, if the result data obtained by calculation is only based on the original data collected by the subsequent node device, it can only reflect the real-time (including real-time location and time) single-point result judgment of the target within the perception range of the subsequent node device; and the result data output by the current node device reflects the direct perception data and result judgment about the target at other locations and other times, or other indirectly related perception data and result judgment, which helps to improve the accuracy and comprehensiveness of the result data of the subsequent node devices, including superposition calculations of the same dimension and correlation references of different dimensions.

[0128] Since there is no master-slave relationship between the node devices in the peer-to-peer computing system, point-to-point transmission can be performed between the node devices. Therefore, for the calculation result corresponding to a certain perception data of a certain target reflected in the result data output by a certain node device, the information is relatively symmetrical in other node devices that receive the result data; other node devices use the received result data as input, combine the perception data of their own sensors, and calculate their own result data. Their own result data naturally covers the received result data and the information reflected by their own sensors, and transmits it to other node devices in the next layer. Therefore, for a certain perception data of a certain target, the information is relatively symmetrical in all node devices, which can be immune to the influence of the calculation process and calculation results of a single node device being tampered with and forged, which leads to the result data. At the same time, it also becomes a means to discover faulty or tampered node devices and node devices with non-compliant performance, fundamentally solving the fundamental hidden dangers of traditional information technology, that is, false information, forged information, and erroneous information caused by information asymmetry, which then become the entry point for fraud and network attacks, and complex comprehensive applications have poor accuracy, too long time consumption, poor credibility, and poor adaptability. Therefore, it can truly become the information infrastructure for comprehensive management of larger areas and the infrastructure of the digital economy. The present invention is different from the blockchain technology that still uses each node to independently calculate, determine the results, and pay attention to the technical solution of original data storage. The present invention focuses on peer-to-peer collaborative computing between node devices. Through peer-to-peer collaborative computing, each node device can adjust its own data processing model (i.e., the algorithm for calculating the result data) and parameters when processing data. The adjustment is the feedback of all node devices on their own adjustments, thereby turning the calculations of all node devices into a whole. Each node device no longer completes the calculation independently, but all node devices complete the calculation together. After the data processing model of the node device is adjusted, it is an objective adjustment that will affect the next data processing.

[0129] The node device is provided with a data acquisition device (in specific implementation, it may include but is not limited to one or more of an image acquisition device, an audio acquisition device, a temperature measurement device, a vibration frequency sensing device, a laser radar, a chemical sensor, and an electromagnetic induction device) and a computing module, and uses the node device of the peer computing system to collect various types of perception data related to the target. Among them, the data acquisition device includes at least one type of sensor for collecting perception data of different corresponding types. The computing module calculates and obtains the result data based on the data processing model. Node devices set at different collection positions (i.e., located at different physical installation positions) collect at least one point sample of the target, and the point sample is the perception data of the corresponding sensor type. The targets include objects already existing in the working area and objects entering the working area.

[0130] In the present invention, the node device deploys a vectorizer for word vectorization of the perception data and the corresponding result processing data, which is used to symbolize (i.e., word vectorization) the various types of perception data collected and the processing result data corresponding to the collected perception data, and generate training word vectors and execution word vectors. Or, further, the node device is deployed with an encoder for encoding the perception data and the corresponding result processing data, and the multimodal perception data and the processing result data corresponding to the collected perception data are encoded based on known encoding rules to obtain symbol encoding corresponding to the encoding rules, so as to realize the symbolization of the multimodal perception data and the processing result data corresponding to the collected perception data (i.e., simplified transcoding), and generate training coded corpus and execution coded corpus; then the node device obtains the symbolized training data (i.e., training word vectors, execution word vectors, or, further, respectively combined with the training coded corpus and the execution coded corpus).

[0131] In this embodiment, for a certain node device, a vectorizer is used to symbolize the various types of perception data collected and the processing result data corresponding to the collected perception data to form a training word vector or an execution word vector, and the training word vector or the execution word vector is processed as the original data to obtain the result data, and the result data is propagated to other node devices; other node devices that receive the result data use the result data as one of the collected original data, and influence the result data of other node devices through the result data. Based on this, multiple node devices in the peer computing system perform collaborative computing, and when the result data of the collaborative computing can determine the target perception result, the target corresponding to the multimodal perception data is perceived to obtain the target perception result, wherein the target perception result is a comprehensive state of one or more combinations of the target's behavior, attribute, state, or relationship between objects, and is represented in the result data.

[0132] When simplifying and transcoding the perception data and the corresponding result processing data, the present invention uses symbols to represent various types of perception data for the target, simplifies the amount of information corresponding to the perception data, and no longer needs to rely on a large and slow database to record massive targets and complex spatiotemporal correlations. The result data output by any node device each time contains the event sequence and details of each dimension over a fairly long period of time covered by the peer computing system.

[0133] The present invention realizes the transmission of target perception results between node devices by transmitting result data between node devices (in the present invention, the result data calculated and transmitted does not include training data and real-time data such as word vectors for word vectorization, word vectors for execution, training coded corpus in symbol combination format, and coded corpus for execution, but the calculation results obtained by calculating the training data and real-time data based on the corresponding data processing mode, that is, when transmitting the result data, the training data and real-time data are not directly transmitted); wherein, the target perception result is a comprehensive state of one or more combinations of the behavior, attribute, state or relationship between objects of the target, and is characterized by the result data. Specifically, for any node device that collects various types of perception data related to the target, the current node device receives the previous result data output by the previous node device, and combines the current perception data collected by the current node device, distributes and transmits it to the node device that meets the calculation requirements through the peer computing system for calculation, and obtains the current result data; and transmits it to the subsequent node device in a collaborative computing manner; and so on, thereby completing the transmission of the target perception result between node devices; and so on, completing the transmission of the target perception result between node devices. That is, in each transmission of result data by a node device, it combines the received data of various types with the current perception data collected by itself, performs calculations based on the data processing model, obtains result data, and then sends the result data to other node devices in the next layer (in most cases, including multiple subsequent node devices, each subsequent node device corresponds to multiple current node devices, and each node device corresponds to multiple preceding node devices; the logical principle of the entire peer-to-peer computing system is the same), that is, the node device completes the transmission of a target perception result.

[0134] In the present invention, the current perception data represents the current state of the target's current behavior, attribute, state or the relationship between objects; the preceding result data represents the preceding through-comprehensive state of the relationship between the target's preceding behavior, attribute, state or objects; the current result data represents the current comprehensive state of the target's current behavior, attribute, state or the relationship between objects, and the current comprehensive state is the combination of the current state and the preceding through-comprehensive state. For example, in the case of a simple single-dimensional single-target, the preceding result data output by the preceding node device is ab (i.e., the result data output by the upper-layer node device of the preceding node device is a, the perception data of the preceding node device is b, and the current result data obtained by the combination is ab), and the current perception data of the current node device is c, then the current result data is abc. The same is true for other complex multi-dimensional and multi-target situations.

[0135] In the present invention, the preceding node device, current node device, and subsequent node device are not necessarily in a fixed order. For example, among the multiple node devices of a peer computing system, three node devices x, y, and z, in a certain transmission of result data, happen to be node device x that transmits result data x to node device y, and node device y that transmits result data y to node device z; and in another transmission of result data, the three node devices x, y, and z do not necessarily have a data transmission relationship; the same is true for the node devices of the entire peer computing system. Correspondingly, in the example given in the present invention, the corresponding transmission path in a single transmission of result data does not mean that the node devices constituting the transmission path are on the same transmission path in each transmission of result data. The transmission path is obtained by induction based on the result data transmission that has occurred, rather than a pre-planned path (including a fixed path and a dynamic path) or a path obtained by calculation. The transmission path of the present invention only represents an inductive result, not its existence.

[0136] Based on the peer-to-peer computing system, there is no master-slave relationship between all node devices. Therefore, the present invention can be implemented without a transmission path, which is different from some existing technologies that must rely on paths to complete event judgment (paths are typical descriptions of a single target, usually what the existing technologies of single-point identification need to rely on to judge events), and achieve event judgment or response from different technical ideas. The node device connects and controls the corresponding execution device, and correspondingly, the response includes the response made by the execution device to which the node device is connected.

[0137] For a certain node device, part of the self-processing result data corresponding to the connected execution device (that is, part of the content in the self-processing result data corresponds to the execution device connected to the control node device) is processed as raw data to obtain result data, and the result data is propagated to other node devices; other node devices that receive the result data use the result data as one of the collected raw data, and affect the result data of other node devices through the result data. Based on this, the node devices in the peer computing system perform collaborative computing as the raw data is acquired and the result data is calculated; the self-processing result data is responded to based on the collaborative computing until the result data calculated by one or more node devices matches the request content corresponding to the self-processing result data (that is, the self-processing result data is regarded as a request, and correspondingly, the request content is formed, such as the need to perform a certain control on a certain execution device, including specific control parameters, etc.).

[0138] In the present invention, the execution device is controlled by symbolic means. Specifically, the self-processing result data represented by the result data is symbolized by a vectorizer deployed on the node device, and converted into a control word vector corresponding to the execution device, and the execution device is controlled by the control word vector; alternatively, the output control word vector is used as a request, and the result data obtained by calculation includes the corresponding control word vector. After the result data is obtained by calculation, the execution device can be directly controlled by the control word vector.

[0139] In step 1) of the present invention, the node devices of the peer computing system are used to collect various types of perception data related to the target; for a certain node device, the perception data of each type collected and the training word vector corresponding to the processing result data corresponding to the collected perception data are obtained. In order to reduce the computing power overhead of optimizing and training a dedicated large language model and improve the training efficiency, in the present invention, the dedicated large language model is segmented, and optimized training is performed by different node devices respectively, and the segmented dedicated large language model is used to generate self-processing result data. In step 1) of the present invention, when optimizing and training using the training word vector, the node device receives the assigned optimization training task and the training word vector and performs corresponding optimization training; the training result is processed as the original data to obtain the result data, and the result data is propagated to other node devices; other node devices that receive the result data use the result data as one of the collected original data, and the result data of other node devices is affected by the result data. Based on this, the node devices in the peer computing system perform collaborative calculations as the original data is obtained and the result data is calculated; the training results are responded to based on the collaborative calculation until the result data calculated by one or more node devices meets the goal of optimization training.

[0140] When the training word vector is used for optimization training using the peer computing system, the training word vector is not included in the result data calculated and transmitted, and the training word vector may not be directly restored. Then, when the training word vector needs to be restored, the present invention can use storage and calling means to restore the training word vector equivalently. Specifically, in the present invention, when the general large language model is optimized and trained using the training word vector, the collected perception data of various types and the processing result data corresponding to the collected perception data are stored in the data storage module of the data acquisition device connected to the corresponding node device. When it is necessary to restore or call and output the perception data and the corresponding processing result data corresponding to the training word vector, the perception data and the corresponding processing result data are called and output as a request to calculate the result data; after the corresponding node device in the peer computing system calculates the result data, the data storage module of the connected data acquisition device is driven to establish a file transfer channel between the node devices of the peer computing system or establish a file transfer channel in other network communication modes, and the perception data and the corresponding processing result data are sent to a specific location to realize calling and output, and can be used for optimization training of the model.

[0141] When it is necessary to train to obtain a general large language model, the present invention can train the general large language model based on the peer computing system. Similarly, in order to reduce the computing power overhead of training to generate a general large language model and improve the training efficiency, in the present invention, the neural network used to train and generate a general large language model is segmented, and the model training is performed through different node devices. In step 1) of the present invention, the training word vector is used as training data, input into the neural network used to generate a general large language model, and the general large language model is trained and generated; when performing model training, the node device receives the assigned model training task and training word vector, and performs corresponding model training; the training result is processed as raw data to obtain result data, and the result data is propagated to other node devices; other node devices that receive the result data use the result data as one of the collected raw data, and the result data of other node devices is affected by the result data. Based on this, the node devices in the peer computing system perform collaborative computing as the raw data is obtained and the result data is calculated; the training results are responded to based on collaborative computing until the result data calculated by one or more node devices meets the goal of model training.

[0142] Correspondingly, in step 2), the node devices of the peer computing system are used to collect various types of perception data related to the target; for a certain node device, the execution word vector formed by the collected various types of perception data is obtained;

[0143] When using a dedicated large language model to generate self-processing result data, the node device receives the assigned generation task and generates the corresponding part of the self-processing result data; the generated result is processed as the original data to obtain the result data, and the result data is propagated to other node devices; other node devices that receive the result data use the result data as one of the collected original data, and influence the result data of other node devices through the result data. Based on this, the node devices in the peer computing system perform collaborative computing as the original data is acquired and the result data is calculated; based on the collaborative computing, the generated result is responded to until the result data calculated by one or more node devices matches the self-processing result data. Among them, the self-processing result data generated by the dedicated large language model is a processing solution for the behavior, attribute, state or relationship between objects at the moment when the objects within the monitoring range are collected by various types of data acquisition devices for perception data; further, in step 2), the corresponding self-processing operation is performed based on the processing solution.

[0144] It can be seen that the present invention utilizes a peer computing system, which can be used to realize the perception of targets, segmentation optimization training of dedicated large language models, segmentation model training of general large language models, and self-processing of result data of dedicated large language models using segmentation, and all are based on the same principle, namely collaborative computing, and the transmission of result data.

[0145] In the present invention, especially in application scenarios of small-scale scenarios and short time periods, simplified transcoding is adopted to obtain coded corpus for training and coded corpus for execution, and based on a peer computing system, the information volume of the original data is integrated into the result data through collaborative calculation. Then, when performing optimization training and self-processing, the collected original data (i.e., step 1) and step 2) collected perception data and corresponding processing result data) do not need to be stored and can be restored, thereby saving storage costs.

[0146] In this embodiment, simplified transcoding and corresponding decoding and restoration are also performed based on a peer-to-peer computing system. Specifically, the node device deploys an encoder for generating coded corpus for training and coded corpus for execution, and a decoder matched with the encoder; wherein the encoder and decoder can generate a coding model for generating coded corpus for training and coded corpus for execution through machine learning, or can be implemented as a hardware encoder and decoder and connected to the node device. Furthermore, when encoding and decoding are required, simplified transcoding and restoration can be performed through the encoder and decoder deployed in the node device, or through the encoder and decoder connected to the corresponding node device.

[0147] Correspondingly, the various types of perception data collected as described in step 1), including but not limited to, the processing result data corresponding to the collected perception data, or the various types of perception data collected as described in step 2), are input into the encoder to generate a training coded corpus or an execution coded corpus in a symbol combination format.

[0148] The training coded corpus or execution coded corpus in the form of a symbol combination of a simple code value + a correction code is used as one of the original data of the calculation result data, and the result data obtained by the calculation is transmitted to the subsequent node device. The subsequent node device receives the result data, uses a decoder to restore the simple code value + the correction code as the training coded corpus or the execution coded corpus from the result data, and then translates the simple code value back to the original code parameter string, and disassembles the original code parameter string into several original code parameters corresponding to the current node device; translates the correction code back to the correction value string, and disassembles the correction value string into several correction values ​​corresponding to the current node device; adds the correction value to the first value of the level paragraph corresponding to the original code parameter, and restores the level of the type of perception data sent by the current node device.

[0149] Based on this, when there is a data restoration request, the result data and the decoder are combined to restore the various types of sensory data collected in step 1), including but not limited to the processing result data corresponding to the collected sensory data, and the various types of sensory data collected in step 2). In specific implementation, data restoration can be used as a request, and the result data obtained by calculation can be directly restored through the encoder and decoder to obtain the various types of sensory data collected in step 1), including but not limited to the processing result data corresponding to the collected sensory data, and the various types of sensory data collected in step 2).

[0150] Specifically, a certain node device is taken as the current node device, and combined with the data transmission of its preceding node device and subsequent node device (the preceding node device and subsequent node device in the present invention are only used to describe the relationship between the current node device and the current node device in the current calculation and data transmission process, and do not mean that there is a necessary relationship between them and the priority relationship), correspondingly, the current node device receives the result data output by other node devices (including the preceding node device), and the subsequent node device receives the result data output by other node devices (including the current node device). For the current node device, the collected perception data is combined with the result data from other node devices (including the preceding node device), the result data of the current node device is calculated, and sent to other node devices (including the subsequent node device). Similarly, the working process of the subsequent node device is the same as that of the current node device, and the preceding node device also receives the result data of the preceding node device of the preceding node device, and performs the same working process as the current node device; that is, the node devices in the peer computing system perform the same working process. Furthermore, the node devices in the peer computing system perform collaborative computing with the collection of perception data and the calculation of result data. The result data output by a certain node device is only received by the subsequent layer of node devices and used as input. The result data of the subsequent layer of node devices will cover the result data of the previous layer of node devices (including the aforementioned node device).

[0151] In a peer-to-peer computing system, all events are processed synchronously, and it is not necessary to explicitly generate staged result outputs such as what events are discovered and what the specific content of the events is; in a peer-to-peer computing system, only the perception of the sensor and the response of the corresponding execution device are clear, and the other intermediate processes are processed by collaborative computing at the same time, that is, during the operation of the present invention, the intermediate process of event discovery is imperceptible, and as the collaborative computing proceeds, the result data of the node device is obtained, and the corresponding execution device automatically responds and executes. Furthermore, the present invention is different from the single-target calculation of the prior art, and is a multi-target state simultaneous calculation, such as tolls. If the single-target calculation of the prior art is used, the specific fee is calculated for a certain vehicle path. The present invention can calculate the fees of all 35,000 vehicles that need to pay for 50,000 vehicles through 5,000 node devices, and simultaneously calculate the paths of 50,000 vehicles, the matching of charging conditions and the fee calculation, and is reflected in the direct driving of the corresponding terminal (that is, the execution device is a payment terminal) to implement non-sensing payment deduction or display the charging QR code according to the corresponding vehicle position (that is, the execution device is a display terminal). During this period, the above-mentioned goals and logics are calculated simultaneously, and there is no calculation mode in which a certain node has a certain vehicle's path information, charging standards, location judgment, and terminal interactive control.

[0152] The present invention is different from the traditional mode. What is transmitted between node devices (including between node devices in a peer computing system) is actually a state representation (such as a representation through a state value), not the state data itself. In the present invention, the state representation can be simplified and transcoded, and some original information can be discarded and compressed under the premise of achieving the event target accuracy. It can also be simplified and transcoded in the original accuracy, so that the node devices can communicate with very small bytes, but ensure that a large number of node devices can accurately calculate and obtain the behavior, attributes, state or relationship between objects of a large number of targets when needed (wherein the acquisition is often unnecessary).

[0153] For example, in a simplified embodiment, target a is sensed by node devices 1, 2, and 3 in sequence, all the way to node device 9; then the target perception data output by node device 2 to node device 3 is target a from node device 1 to node device 2, that is, 1-2, which can be described by a state value in the matrix. Subsequently, node device 3 also senses target a, and the perception result of node device 3 is added in a matrix manner, that is, 1-2-3, and sent to the target node device. 1-2-3 is still a state value in the matrix. Similarly, the target perception data transmitted by node device 9 to the next group of node devices is target a from node device 1 to node device 9, that is, 1-2-3-4-5-6-7-8-9, which is still just a state value in the matrix.

[0154] In the case of full coverage, the path information involved in the node devices is often multi-dimensional. Correspondingly, the node devices will receive multiple result data. For example, node device x may receive the tracking sequences of target a corresponding to paths such as path 123, path 825, path abc, path dbm, and path lmn. Node device x aggregates these result data into a matrix for expression. Therefore, although these node devices discover target a from their respective different paths, at the node device x which is the collection point of all paths, the target perception results of all node devices for target a will be expressed in the form of a matrix at the position of node device x, and the expression result is still a state value.

[0155] During specific implementation, different node devices may collect data of different types due to different sensor types. Correspondingly, the types of perception data collected by different node devices are also not exactly the same. Furthermore, if the current node device does not have one or more types of sensors set by the previous node device, the current node device will still take the previous result data containing the corresponding type of perception data sent by the previous node device as input, and synchronously process it with the perception data collected by the sensor of the current node device to avoid the loss of some types of perception data during the result data transmission process, thereby affecting the integrity of the result data.

[0156] In the present invention, the automatic control that the execution device needs to perform can be regarded as a request command. In the present invention, the response to the request command includes a variety of different situations such as "demand-execution", "request-response" or other situations. In the present invention, when the result data calculated by one or more node devices in the peer computing system matches the request content corresponding to the self-processing result data, the result corresponding to the request content (for example, the result after self-processing based on the self-processing result data) is characterized in the result data output by one or more node devices according to the preset conditions or the output of the data processing model deployed in advance or the program deployed on the node device; if it is determined that the current node device needs to respond to the request command, the current node device will send a control instruction to the execution device connected to the current node device according to the calculated result data; that is, the "demand-execution" situation. When all the execution devices connected to all the node devices that need to respond have completed the response action, the self-processing operation is completed. When the response to the request command is "request-response", that is, the request content corresponding to the request command issued by the access terminal requires the peer computing system to feedback the response content, then the obtained response content is transcribed through multiple semantic modules to obtain one or several data information that matches the requester's needs, and is responded by one or more output terminals (execution devices).

[0157] In order to ensure that the data source and the calculation process are further credible, in the present invention, all node devices encrypt the result data obtained by their calculations based on the encryption consensus mechanism to obtain the encryption result, and then send the encryption result to other node devices. The encryption consensus mechanism includes one or more consensus mechanisms, and different consensus mechanisms correspond to changes in the encryption algorithm structure and parameters of the node device.

[0158] Node devices communicate with each other using standard-sized data packets (i.e., result data or calculation results). In the present invention, the node devices of the peer-to-peer computing system are similar to human neurons. Each neuron does not transmit specific data that directly describes external events. Similarly, the node devices do not output raw data, but process the raw data obtained by the connected sensors and data acquisition devices into standard-sized data packets (i.e., result data or calculation results, similar to the nerve impulses of neurons) according to their own data processing models (similar to the biological characteristics of nerve cells). The amount of information contained in a single data packet is not enough to restore any event and target information. The calculation results, multi-dimensional data matrix elements, and physical space and facility correspondences on the entire peer-to-peer computing system must be jointly calculated to obtain a definite result. The collaborative calculation is not very dependent on the data output by a few node devices, and the collaborative calculation simultaneously processes the needs received or initiated by all node devices. It is a collaborative verification calculation of super-multi-dimensional related information, which can fundamentally change the nature of traditional information technology that is sensitive to single-point security.

[0159] In order to ensure the integrity of data and the effective execution of collaborative computing, in the present invention, the peer computing system deploys a QoS mechanism, and the QoS mechanism is to give priority to ensuring the transmission quality of the result data between node devices.

[0160] In the specific implementation, the networking mode of the peer computing system is one or a combination of 4G mode, 5G mode or MESH mode, so as to adapt to different application scenarios, comprehensively implement feasibility, cost considerations and other factors, and realize the optimal solution. Among them, the MESH mode is based on the LTE standard, and communicates at the physical layer of the LTE standard; it carries data with a customized frame structure and interacts with a dedicated wireless communication protocol. The frame structure is customized to be suitable for the calculation of the peer computing system, and the proprietary wireless communication protocol developed for the calculation of the urban agglomeration peer computing system is adopted, which can further improve its security and reliability. In addition, the wireless algorithm fully adapts to the multipath channel environment based on the consensus mechanism control required for the calculation of the peer computing system. The communication distance in the city is 100 meters-10 kilometers, and the omnidirectional antenna is used in the field. Under the condition of efficient transmission of 120 kilometers, it can also achieve efficient transmission at a distance of 120 kilometers. In this embodiment, the communication distance of the Mesh network is: the distance between node devices indoors is 50-150 meters, the distance between node devices outdoors is 50 meters-120 kilometers, and the number of node devices that can be accessed by each node is 65535. In addition, when networking in 4G mode or 5G mode, there is no limit on the communication distance, and the number of accessible node devices depends on the computing power of the computing chip and the communication delay.

[0161] In a peer-to-peer computing system, for a certain point sample of a certain object, the result data transmitted from the node device that collects the point sample to other node devices can enable subsequent node devices to adjust their perceptual attention according to the characteristics of the point sample (it is not necessary to include the characteristics of the point sample in the result data, but the characteristics of the point sample participate in the calculation of the previous node device, so that when the result data of the previous node device is used as the input of the data processing model of the subsequent node device, the data processing model of the subsequent node device can achieve the effect of adjusting perceptual attention in the calculation); or, report the characteristics of the point sample for subsequent node devices to adjust their perceptual attention (directly express the characteristics of the point sample in the result data). If other subsequent node devices do not detect the characteristics of the point sample, but it can be determined from the characteristics of other point samples that the characteristics of the point sample that cannot be detected still belong to the object, then the characteristics of the point sample that cannot be detected will continue to be expressed in the result data of the current node device and transmitted to other node devices. For example, the previous node device perceives the color of object A, and the current node device does not perceive the color of object A. However, it can be determined from the perception data of other node devices that object A exists in addition to other objects. Then the color of object A that has not been perceived is still expressed in the result data of the current node device.

[0162] In this embodiment, the method of reporting the characteristics of the point sample for subsequent node devices to adjust the perception attention is: adjusting the parameters of the data processing model of the subsequent node device according to the characteristics of the point sample provided by the previous node device, so that the subsequent node device improves the computing power of identifying the characteristics of the point sample; or, the subsequent node device uses the perception attention model to match the characteristics of the received point sample to adjust the computing power. Among them, the above-mentioned "feature" is different from the meaning of "feature recognition" in the prior art. The "feature recognition" in the prior art usually refers to information that can determine the identity of the target, while the "feature" of the present invention represents a kind of perceived perception data belonging to the object, such as coordinates, the color of the object, etc., and the "non-specific feature recognition" of the object cannot be directly completed by the "feature" of single-point perception. The "non-specific feature recognition" described is different from the "recognition" in the usual sense in terms of strict conceptual definition. The "recognition" in the usual sense refers to the concreteness of the target or the specific identity information of the target, such as who it is (including name, specific information indicating the identity of the target), what it is (such as a car, a person, etc.). The "identification" in the "non-specific feature identification" described in the present invention means that each unique target (i.e., an object that already exists in the working area of ​​the vehicle, an object that enters the working area) is identified as itself; that is, for a certain object to be identified, its existence is unique. After the present invention implements "non-specific feature identification", it is determined that the object to be identified (i.e., a target that has not been identified or confirmed) is itself, rather than other objects to be identified. The result of "non-specific feature identification" does not need to determine the specific features of the object to be identified, and does not need to determine the identity information or concrete image of the object to be identified. For example, for a person, it is regarded as the object to be identified A, and for an object, it is regarded as the object to be identified B. Then, after implementing "non-specific feature identification", it is not necessary to identify the object to be identified A as a person and who is the specific identity, and it is not necessary to identify the object to be identified B as an object and what the specific object is; instead, it is necessary to determine that the object to be identified A is the object to be identified A itself, and the object to be identified B is the object to be identified B itself. Then, corresponding services or controls can be performed for the object to be identified A or the object to be identified B.

[0163] In this embodiment, the method for reporting the characteristics of the point sample for subsequent node devices to adjust perceptual attention is: adjusting the parameters of the data processing model of the subsequent node device according to the result data expressing the characteristics of the point sample provided by the previous node device (in the present invention, the characteristics of the point sample itself are usually not provided, but the characteristics of the point sample are expressed in the result data), or the characteristics of the point sample (that is, the characteristics of the point sample itself), so that the subsequent node device improves the computing power for identifying the characteristics of the point sample; or, the subsequent node device uses the perceptual attention model to match the characteristics of the received point sample or the result data expressing the characteristics of the point sample to adjust the computing power.

[0164] When a node device processes the result data output by several previous node devices, based on the data processing model, when the objects described by several previous node devices can be determined as the same target through some common point sample features, the point sample features and other information described by each node device are merged into the same target. For example, point sample features in physical spaces that are almost completely overlapped at the same time can be determined to be the same target.

[0165] In the present invention, the data acquisition device of the node equipment includes one or several combinations of an image acquisition device, an electromagnetic induction device, a temperature measurement device, a vibration frequency sensing device and a laser radar. The data collected by the above devices (i.e., one or several combinations of the image acquisition device, the electromagnetic induction device, the temperature measurement device, and the vibration frequency sensing device) and the three-dimensional point cloud collected by the laser radar or the point cloud generated according to the images collected by multiple image acquisition devices are jointly calculated to obtain three-dimensional points with data; the image color, contour, line, reflectivity, motion trend, electromagnetic characteristics, temperature, temperature change trend, vibration frequency, and vibration frequency change trend based on two-dimensional perception are used as additional attributes of the corresponding three-dimensional points to form a three-dimensional point cloud with attributes; combined with electromagnetic induction, temperature law, vibration frequency change characteristics, motion correlation (different motion correlations presented by different materials such as ropes and cloth), and reflectivity, the correspondence between each area of ​​the three-dimensional point cloud with attributes and each part or associated parts of the 3D appearance of the object to be identified is determined. This embodiment uses the attributes of the attributed three-dimensional point cloud and their correlation to determine the relationship between each point, the correspondence between each area to which each related point belongs and each part or associated part of the 3D appearance of the object to be identified, so as to more accurately determine the point sample features belonging to the object to be identified.

[0166] When the result data received by the node device indicates that the flag used by the current node device to identify the object before the current node device receives the result data is different from the flag used by other node devices to identify the object, and the flags assigned to the object by other node devices are updated, the flag used by the current node device to identify the object before the current node device receives the result data is converted. Specifically, the method for converting the flag used by the current node device to identify the object before the current node device receives the result data is:

[0167] The mark used by the current node device to identify the object before the current reception of result data is replaced with the latest mark allocated to the object by other node devices; this is a relatively simple implementation method provided by the present invention.

[0168] Alternatively, the conversion relationship between the flag used by the current node device to identify the object before the current reception of the result data and the flag assigned to the object by other node devices after the update is recorded, and the conversion is performed when it is necessary to reference the current node device's current reception of the result data; this is a relatively complex implementation method provided by the present invention.

[0169] Alternatively, the node device deploys a conversion model to perform corresponding conversions on the flags of multiple objects according to the input original data or result data; this is a more complex implementation method provided by the present invention.

[0170] The above embodiments are only used to illustrate the present invention, and are not used to limit the present invention. As long as they are based on the technical essence of the present invention, changes and modifications to the above embodiments will fall within the scope of the claims of the present invention.

Claims

1. A self-processing method based on perceiving the real world, characterized in that: The steps include: 1) Optimize the training steps: use various types of data acquisition devices deployed in the working area to collect various types of corresponding types of perception data, and the collected types of perception data cover objects within the monitoring range of the corresponding types of data acquisition devices; obtain processing result data corresponding to the collected perception data; Symbolizing the various types of sensory data collected, including but not limited to the sensory data, and the processing result data corresponding to the sensory data collected, to form a training word vector with natural semantics, wherein the training word vector describes the multimodal sensory data and the processing result data corresponding to the sensory data collected, and inputting the training word vector as training data into a general large language model, optimizing and training the general large language model, and obtaining a special large language model; 2) Self-processing step: Utilize various types of data acquisition devices deployed in the working area to collect various corresponding types of perception data, symbolize the collected perception data of each type, and form execution word vectors with natural semantics. The execution word vectors describe the multimodal perception data, and use the execution word vectors as real-time data to input into a dedicated large language model to generate self-processing result data with natural semantics, and perform corresponding self-processing operations based on the self-processing result data.

2. The self-processing method based on perceiving the real world according to claim 1, characterized in that: The self-processing result data generated by the dedicated large language model describes, including but not limited to, interaction data and / or task strategies; When the self-processing result data describes the interactive device interaction data and the task strategy, the interactive device is controlled to output the interactive data through the matching task strategy; When the self-processing result data describes the task strategy, the task strategy is used to control the corresponding object to perform the corresponding operation.

3. The self-processing method based on perceiving the real world according to claim 2, characterized in that: The task strategy includes control instructions and instruction sequences, and controls corresponding objects to perform corresponding operations based on the task strategy.

4. The self-processing method based on perceiving the real world according to claim 3, characterized in that: The object includes or is provided with at least one object, and various working parameters of the object are obtained as perception data; the object responds to the various working parameters included in the processing scheme to control the object.

5. The self-processing method based on perceiving the real world according to claim 2, characterized in that: The interactive data includes but is not limited to voice interactive data, image interactive data, and text interactive data, and data interaction is performed through corresponding types of interactive devices.

6. The self-processing method based on perceiving the real world according to claim 1, characterized in that: Perception data also includes but is not limited to laws and regulations, implementation standards, specifications, and custom constraints; the processing result data corresponding to the collected perception data includes but is not limited to control instructions and interactive information.

7. The self-processing method based on perceiving the real world according to claim 1, characterized in that: In step 1), taking functional scenarios as units, various types of perception data collected in different functional scenarios and training word vectors corresponding to the processing result data corresponding to the collected perception data are used to input into a general large language model for optimization training to generate a special large language model; the special large language model generates the processing result data based on the execution word vector; wherein, there is no need to predefine the functional scenarios.

8. The self-processing method based on perceiving the real world according to claim 7, characterized in that: In step 1), when there are multiple independent functional scenarios with the same functions, firstly, several of the multiple functional scenarios are automatically selected according to the selection rules, and then the selected functional scenarios are input into the general large language model for optimization training.

9. The self-processing method based on perceiving the real world according to claim 8, characterized in that: Further, after automatically selecting several functional scenarios, the training word vectors are output first, or when the functional scenarios are used to optimize the training of the general large language model, the collected perception data of various types and the processing result data corresponding to the collected perception data are stored, and then called and output, and then the selected functional scenarios are prioritized manually based on the output training word vectors or perception data and the corresponding processing result data; And / or, by manually setting, set the starting and ending points of the training word vector input for the functional scenario optimization training, wherein the starting and ending points of the training word vector input are respectively the state where the training word vector starts inputting and the state where the training word vector ends inputting.

10. The self-processing method based on perceiving the real world according to claim 7, characterized in that: Step 1) specifically comprises the following steps: using various types of data acquisition devices deployed in the working area to continuously acquire various corresponding types of perception data, wherein the continuously acquired perception data of various types include perception data of objects in the working area covered by the monitoring range of the corresponding type of data acquisition device that are acquired in chronological order; symbolizing the continuously acquired perception data of various types and the processing result data corresponding to the acquired perception data, and converting them into training word vectors with natural semantics; inputting the training word vectors into a general large language model for optimization training, and generating a dedicated large language model.

11. The self-processing method based on perceiving the real world according to claim 1, characterized in that: A three-dimensional global model is obtained by three-dimensional modeling, wherein the three-dimensional global model includes a three-dimensional object model representing the object; In step 1), various types of corresponding virtual acquisition devices are deployed in the three-dimensional global model; Using a virtual acquisition device to perform in-model acquisition on a three-dimensional object model to obtain in-model perception data; inputting in-model training word vectors corresponding to the in-model perception data into a dedicated large language model to generate in-model self-processing result data, and generating an automated script corresponding to the in-model self-processing result data; The self-processing result data in the model is the model-in-processing solution for the behavior, attribute, state or relationship between objects of the three-dimensional object model at the moment when the model-in-perception data is collected; the self-processing result data in the model is used as the model-in-processing result data corresponding to the model-in-perception data; The three-dimensional object model corresponding to the processing result data in the model is controlled by automated scripts or manually controlled to execute the model processing solution in the three-dimensional global model, thereby optimizing the training of the dedicated large language model.

12. The self-processing method based on perceiving the real world according to claim 10, characterized in that: Delimiting a modeling area corresponding to the working area; performing three-dimensional modeling on real objects in the modeling area to construct a three-dimensional global model; the three-dimensional global model includes a three-dimensional object model corresponding to the real object; Correspondingly, in step 1), corresponding to various types of data acquisition devices in the working area, corresponding various types of virtual acquisition devices are deployed in the three-dimensional global model; the virtual acquisition devices are used to perform in-model acquisition on the three-dimensional object model to obtain the perception data within the model.

13. The self-processing method based on perceiving the real world according to claim 1, characterized in that: The symbolization described in step 1) and step 2) further includes, by encoding, converting the various types of sensory data described in step 1) including but not limited to the collected sensory data, the processing result data corresponding to the collected sensory data, and the various types of sensory data collected in step 2) to obtain training coded corpus and execution coded corpus in a symbol combination format for simplifying transcoding of a multidimensional matrix, specifically: Different types of perception data belonging to different objects collected by different types of data collection devices and the processing result data corresponding to the collected perception data are simplified and transcoded based on transcoding rules to obtain matching symbol simple codes; a perception multidimensional matrix is ​​constructed for the symbol simple codes corresponding to all current perception data collected by the data collection device; The symbol combination format includes, but is not limited to, one or more combinations of texts, characters, super-large base sequences, known symbols of non-natural languages, and user-defined symbols of various natural languages; In step 1), the training encoded corpus is combined with the training word vector as training data, and input into the general large language model to optimize the general large language model; In step 2), the execution coded corpus is combined with the execution word vector as real-time data and input into a dedicated large language model to generate self-processing result data with natural semantics.

14. The self-processing method based on perceiving the real world according to claim 13, characterized in that: Exhaustively enumerate all symbol simple codes and the perceptual multidimensional matrices they constitute, and assign a certain simple code value to each perceptual multidimensional matrix; construct an M-bit N-ary symbol sequence, where each bit of the N-ary symbol sequence is assigned one of K different symbols; match each specific value of the M-bit N-ary sequence with each simple code value one by one; wherein the value of M×N is greater than the number of simple code values.

15. The self-processing method based on perceiving the real world according to claim 13, characterized in that: The transcoding rule is as follows: different types of perception data belonging to different objects collected by different data collection devices and processing result data corresponding to the collected perception data are predefined into several levels, and all levels are divided into several level sections; a corresponding level section identifier is assigned to each level section, and a predefined type identifier is assigned to each type of perception data, and the type identifier + level section identifier constitutes the original code parameter; the difference between the level of a type of perception data and the first value of the level section to which it belongs is set as a correction value; Arrange multiple groups of original code parameters that need to be simultaneously input into a general large language model or a special large language model to form an original code parameter string, arrange the correction values ​​to form a correction value string, convert the original code parameter string into a matching simple code value, convert the correction value string into a matching correction code, and use the simple code value + correction code as a coded corpus for training and a coded corpus for execution.

16. The self-processing method based on perceiving the real world according to claim 15, characterized in that: All original code parameter strings of different types of data acquisition devices are exhaustively listed, and a certain simple code value is assigned to each original code parameter string; all correction value strings of different types of data acquisition devices are exhaustively listed, and a certain correction code is assigned to each correction value string.

17. The self-processing method based on perceiving the real world according to claim 16, characterized in that: The word length of the simplified code value is smaller than the word length of the original code parameter string, and the word length of the correction code is smaller than the word length of the correction value string.

18. The self-processing method based on perceiving the real world according to claim 17, characterized in that: If the number of exhaustively enumerated original code parameter strings and correction value strings is less than Q, where Q is the set number of symbols; then the simplified code value and the correction code are both represented by one symbol, and each original code parameter string and correction value string matches one symbol respectively; If the number of exhaustive original code parameter strings and correction value strings is greater than the set number of symbols, each original code parameter string and correction value string matches at least two bits of symbols, and multiple bits of symbols are arranged to form a symbol string; wherein the symbol string includes a number of bits of symbols, and the number of original code parameter strings matched by the symbol string is P = Q n , where n is the number of symbol bits, n = 1, 2, 3, ...

19. The self-processing method based on perceiving the real world according to claim 18, characterized in that: The number of bits of the symbol string is determined according to the number of exhaustively enumerated original code parameter strings and correction value strings, so that the number of original code parameter strings and correction value strings matched by the symbol string is not less than the number of exhaustively enumerated original code parameter strings and correction value strings.

20. The self-processing method based on perceiving the real world according to claim 1, characterized in that: The data acquisition device includes one or a combination of an image acquisition device, an electromagnetic induction device, a temperature measuring device, a vibration frequency sensing device and a laser radar. The data collected by the above devices and the three-dimensional point cloud collected by the laser radar or the point cloud generated according to the images collected by multiple image acquisition devices are jointly calculated to obtain three-dimensional points with data; the image color, contour, line, reflectivity, motion trend, electromagnetic characteristics, temperature, temperature change trend, vibration frequency, and vibration frequency change trend based on two-dimensional perception are used as additional attributes of the corresponding three-dimensional points to form a three-dimensional point cloud with attributes; combined with electromagnetic induction, temperature law, vibration frequency change characteristics, motion correlation, and reflectivity, the correspondence between each area of ​​the three-dimensional point cloud with attributes and each part or associated parts of the 3D appearance of the object is determined.

21. The self-processing method based on perceiving the real world according to claim 1, characterized in that: If in step 1), new training word vectors are obtained using perception data of different working areas, different objects, and different modalities, and the dedicated large language model is optimized and trained, then the dedicated large language model is optimized and trained to enable the dedicated large language model to increase adaptation to perception data of different objects and different modalities.

22. The self-processing method based on perceiving the real world according to any one of claims 1 to 21, characterized in that: Setting up a peer-to-peer computing system, the peer-to-peer computing system includes a plurality of node devices, and there is no primary-secondary relationship between all the node devices; using the node devices of the peer-to-peer computing system to collect various types of perception data related to the target, the target including an object already existing in the working area and an object entering the working area; The node device deploys a vectorizer for symbolizing the perception data; for a certain node device, the vectorizer is used to symbolize the various types of perception data collected and the processing result data corresponding to the collected perception data to form a training word vector or an execution word vector, and the training word vector or the execution word vector is processed as the original data to obtain the result data, and the result data is propagated to other node devices; other node devices that receive the result data use the result data as one of the collected original data, and influence the result data of other node devices through the result data; Based on this, multiple node devices in the peer computing system perform collaborative computing. When the result data of the collaborative computing can determine the target perception result, the target corresponding to the multimodal perception data is perceived to obtain the target perception result, wherein the target perception result is a comprehensive state of one or more combinations of the target's behavior, attributes, state, or the relationship between objects, and is represented in the result data.

23. The self-processing method based on perceiving the real world according to claim 22, characterized in that: The node device connects to and controls the corresponding execution device; For a certain node device, a portion of the self-processing result data corresponding to the connected execution device is processed as original data to obtain result data, and the result data is transmitted to other node devices; other node devices that receive the result data use the result data as one of the collected original data, and influence the result data of other node devices through the result data; The node devices in the peer-to-peer computing system perform collaborative computing as the original data is acquired and the result data is calculated; Respond to the self-processing result data based on collaborative computing until the result data calculated by one or more node devices matches the request content corresponding to the self-processing result data; The self-processing result data represented by the result data is symbolized by a vectorizer deployed on the node device, and converted into a control word vector corresponding to the execution device, and the execution device is controlled by the control word vector; or, the output control word vector is used as a request, the result data obtained by calculation includes the corresponding control word vector, and the execution device is directly controlled by the control word vector.

24. The self-processing method based on perceiving the real world according to claim 22, characterized in that: In step 1), various types of perception data related to the target are collected using node devices of the peer computing system; for a certain node device, the collected perception data of various types and the training word vector formed corresponding to the processing result data corresponding to the collected perception data are obtained; When using training word vectors for optimization training, the node device receives the assigned optimization training tasks and training word vectors, and performs corresponding optimization training; processes the training results as raw data to obtain result data, and propagates the result data to other node devices; other node devices that receive the result data use the result data as one of the collected raw data, and influence the result data of other node devices through the result data; The node devices in the peer-to-peer computing system perform collaborative computing as the original data is acquired and the result data is calculated; Respond to the training results based on collaborative computing until the result data calculated by one or more node devices meets the goal of optimizing the training.

25. The self-processing method based on perceiving the real world according to claim 22, characterized in that: When optimizing and training the general large language model using the training word vector, the collected perception data of various types and the processing result data corresponding to the collected perception data are stored in the data storage module of the data collection device connected to the corresponding node device; The perception data and the corresponding processing result data are called and output as requests to calculate the result data. After calculating the result data, the corresponding node device in the peer computing system drives the data storage module of the connected data acquisition device to establish a file transfer channel between the node devices of the peer computing system or to establish a file transfer channel in other network communication modes, and sends the perception data and the corresponding processing result data to a specific location.

26. The method of self-processing based on perceiving the real world according to claim 22, characterized in that: In step 1), various types of perception data related to the target are collected using node devices of the peer computing system; for a certain node device, the collected perception data of various types and the training word vector formed corresponding to the processing result data corresponding to the collected perception data are obtained; The training word vector is used as training data and input into a neural network for generating a general large language model, and the general large language model is trained and generated; when performing model training, the node device receives the assigned model training task and the training word vector and performs corresponding model training; the training result is used as raw data for processing to obtain result data, and the result data is propagated to other node devices; other node devices that receive the result data use the result data as one of the collected raw data, and influence the result data of other node devices through the result data; The node devices in the peer-to-peer computing system perform collaborative computing as the original data is acquired and the result data is calculated; Respond to the training results based on collaborative computing until the result data calculated by one or more node devices meets the objectives of model training.

27. The self-processing method based on perceiving the real world according to claim 24, characterized in that: In step 2), various types of perception data related to the target are collected by using node devices of the peer computing system; for a certain node device, an execution word vector formed by the collected various types of perception data is obtained; When using a dedicated large language model to generate self-processing result data, the node device receives the assigned generation task and generates the corresponding part of the self-processing result data; the generated result is processed as the original data to obtain the result data, and the result data is transmitted to other node devices; other node devices that receive the result data use the result data as one of the collected original data, and influence the result data of other node devices through the result data; The node devices in the peer-to-peer computing system perform collaborative computing as the original data is acquired and the result data is calculated; The generated results are responded to based on collaborative computing until the result data calculated by one or more node devices matches the self-processing result data.

28. The self-processing method based on perceiving the real world according to claim 27, characterized in that: For any node device that collects various types of perception data related to the target, the current node device receives the previous result data output by the previous node device, and combines it with the current perception data collected by the current node device, distributes it through the peer computing system and transmits it to the node device that meets the computing requirements for calculation, and obtains the current result data; and transmits it to the subsequent node device in a collaborative computing manner; and so on, thereby completing the transmission of the target perception results between the node devices; The target perception result is a comprehensive state of one or more combinations of the target's behavior, attributes, state, or relationship between objects, and is represented in the result data; The current perception data represents the current behavior, attributes, state, or current state of the relationship between objects; The predecessor result data represents the predecessor behavior, attribute, state or the relationship between objects of the target predecessor. The current result data represents the current comprehensive state of the target's current behavior, attribute, state or the relationship between objects. The current comprehensive state is the combination of the current state and the previous comprehensive state.

29. The method of self-processing based on perceiving the real world according to claim 28, characterized in that: The self-processing result data generated by the dedicated large language model is a processing solution for the behavior, attributes, state or relationship between objects within the monitoring range at the moment when the perception data is collected by various types of data acquisition devices; in step 2), the corresponding self-processing operation is performed based on the processing solution.

30. The self-processing method based on perceiving the real world according to claim 22, characterized in that: The node device is deployed with an encoder for generating coded corpus for training and coded corpus for execution, and a decoder matching the encoder; Inputting the various types of sensory data collected in step 1), including but not limited to, the processing result data corresponding to the collected sensory data, or the various types of sensory data collected in step 2), into an encoder to generate a training coded corpus or an execution coded corpus in a symbol combination format; The training coded corpus or execution coded corpus in the symbol combination form of the simple code value + the correction code is used as one of the raw data of the calculation result data, and the result data obtained by the calculation is transmitted to the subsequent node device; The subsequent node device receives the result data, uses a decoder to restore the simple code value + correction code as the training coded corpus or the execution coded corpus from the result data, and then translates the simple code value back to the original code parameter string, and disassembles the original code parameter string into a number of original code parameters corresponding to the current node device; translates the correction code back to the correction value string, and disassembles the correction value string into a number of correction values ​​corresponding to the current node device; adds the correction value to the first value of the level section corresponding to the original code parameter, and restores the level of the type of perception data sent by the current node device; Based on this, when there is a data restoration request, the result data is combined with the decoder to restore the various types of perception data collected in step 1), including but not limited to the processing result data corresponding to the collected perception data, and the various types of perception data collected in step 2).

31. The self-processing method based on perceiving the real world according to claim 30, characterized in that: In step 1), the sensory data of various types collected and the processing result data corresponding to the collected sensory data corresponding to the training coded corpus are not stored; In step 2), the perception data of various types corresponding to the encoding corpus for execution are not stored.

32. The method of self-processing based on perceiving the real world according to claim 22, characterized in that: When the result data calculated by one or more node devices in the peer computing system matches the request content corresponding to the self-processing result data, the result corresponding to the request content is represented in the result data output by the one or more node devices according to the preset conditions or the output of the pre-deployed program or the data processing model deployed on the node device; If it is determined that the current node device needs to respond to the request command, the current node device will send a control instruction to the execution device connected to the current node device according to the calculated result data; when the execution devices connected to all the node devices that need to respond have completed the response action, the self-processing operation is completed.