A scene difference information storage method and system

Abstract

The application relates to the technical field of intelligent sensing systems, in particular to a scene difference information storage method and system, which comprises the following steps: identifying a plurality of target objects from a scene image set collected by a space intelligent machine and acquired space structure dynamic information; for any target object, adopting a preset difference identification algorithm based on the space structure dynamic information to identify difference information of the target object in several dimensions; inputting the difference information of each target object in different dimensions into a preset world model; outputting a target difference feature vector corresponding to the scene image set; and storing the target difference feature vector corresponding to the scene image set and preset original scene data in the space intelligent machine; the application can realize real-time and accurate scene difference analysis by performing difference analysis and storage on a complex scene through high-dimensional space data, and is beneficial to efficient query and reasoning of subsequent tasks.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent perception systems, in particular to a scene difference information storage method and system. BACKGROUND

[0002] Current intelligent environment perception systems usually rely on real-time image processing technology to monitor environmental changes, but there are generally problems such as poor utilization of historical data, low monitoring accuracy, etc., and it is not possible to quickly and accurately query and utilize key information of environmental changes from historical data. In addition, traditional big data processing methods, such as access log analysis method or EventSourcing paradigm, although can realize event recording and historical backtracking, usually only process structured or semi-structured data, lack of in-depth understanding of high-dimensional spatial semantic information, and are difficult to perform complex scene analysis and real-time reasoning, affecting the query processing accuracy and query efficiency of scene tasks. SUMMARY

[0003] In view of the above technical problems, the present application provides a scene difference information storage method and system, which analyzes the differences of complex scenes through high-dimensional spatial data and stores difference information, can realize real-time and accurate scene difference analysis and significantly reduce the data storage amount, and is beneficial to efficient query and reasoning of subsequent tasks.

[0004] According to a first aspect of the present application, a scene difference information storage method is provided, comprising the following steps:

[0005] S100, according to the scene image set of the target area collected by the pre-deployed spatial intelligent machine in the preset time period and the obtained spatial structure dynamic information, a plurality of target objects are identified from the scene image set based on a plurality of preset category objects; the plurality of preset category objects include scene background, personnel, object, robot and IOT device, scene behavior event and scene macroscopic law.

[0006] S200, for any target object, the difference information of the target object in a plurality of dimensions is identified by using a preset difference recognition algorithm based on the spatial structure dynamic information; the difference information is the semantic information corresponding to the difference of the target object in any dimension; wherein the difference recognition algorithm corresponding to the scene background, personnel, object, robot and IOT device, scene behavior event and scene macroscopic law is semantic segmentation and scene recognition algorithm, human pose estimation and behavior recognition algorithm, target detection and tracking algorithm, device state recognition algorithm, event detection algorithm and statistical analysis algorithm.

[0007] S300, the difference information of each target object in different dimensions is input into a preset world model, and a target difference feature vector corresponding to the scene image set is output by the preset world model.

[0008] S400, store the target difference feature vector corresponding to the scene image set and the preset original scene data into the space intelligent machine.

[0009] According to the second aspect of the present application, a scene difference information storage system is provided, which comprises:

[0010] The first identification module is configured to identify a plurality of target objects from the scene image set based on a plurality of preset category objects according to the scene image set of the target region collected by the pre-deployed space intelligent machine within a preset time period and the obtained space structure dynamic information; the plurality of preset category objects include scene background, personnel, object, robot and IOT device, scene behavior event and scene macroscopic law.

[0011] The second identification module is configured to identify the difference information of the target object in a plurality of dimensions by using a preset difference identification algorithm based on the space structure dynamic information for any target object; the difference information is the semantic information corresponding to the difference of the target object in any dimension; wherein the difference identification algorithms corresponding to the scene background, personnel, object, robot and IOT device, scene behavior event and scene macroscopic law are semantic segmentation and scene identification algorithm, human pose estimation and behavior identification algorithm, target detection and tracking algorithm, device state identification algorithm, event detection algorithm and statistical analysis algorithm.

[0012] The processing module is configured to input the difference information of each target object in different dimensions into a preset world model, and output a target difference feature vector corresponding to the scene image set through the preset world model.

[0013] The storage module is configured to store the target difference feature vector corresponding to the scene image set and the preset original scene data into the space intelligent machine.

[0014] The present application has at least the following beneficial effects:

[0015] 1. Based on the scene image set of the target region collected by the pre-deployed space intelligent machine within a preset time period, a plurality of target objects are identified from the scene image set, and through object decomposition, it is beneficial to individually analyze the difference of each independent object, and the difference analysis quality can be improved.

[0016] 2. By using the corresponding difference identification algorithm to identify the difference information of each target object in a plurality of dimensions, and by using different difference identification algorithms for different target objects, the difference information of different target objects in multiple dimensions can be accurately identified, so that the identified difference information is more comprehensive, which is beneficial to improve the subsequent accurate query.

[0017] 3. By inputting the difference information of each target object in different dimensions into the preset world model, and outputting the target difference feature vector corresponding to the scene image set through the preset world model, the characteristics of the world model that can uniformly quantify the difference information of multiple dimensions and efficiently process high-dimensional spatial data are utilized, which is conducive to real-time and accurate scene difference analysis of complex scenes in this application.

[0018] 4. By associating and storing the target difference feature vector corresponding to the scene image set with the preset original scene data in the spatial intelligent machine, the scene change information is stored in the form of difference feature vectors, which effectively avoids the repeated storage of a large amount of similar or redundant original scene data, significantly reduces the data storage space occupation, and only needs to be retrieved and reasoned for the difference information in subsequent scene queries, which greatly reduces the complexity of data query and processing and improves the efficiency of real-time response and query of the system. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 A flowchart of a method for storing scene difference information provided in Embodiment 1 of the present invention;

[0021] Figure 2 This is a schematic diagram of the structure of the scene difference information storage system provided in Embodiment 2 of the present invention. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] Example 1

[0024] Embodiment 1 of the present invention provides a method for storing scene difference information, such as Figure 1 As shown, the storage method includes the following steps:

[0025] S100, according to the scene image set of the target area collected by the pre-deployed space intelligence machine in the preset time period and the obtained space structure dynamic information, a plurality of target objects are identified from the scene image set based on a plurality of preset category objects; it can be understood that: the space intelligence machine collects images in real time through the camera data, and fuses local sensors and its own environment perception and space modeling functions, continuously monitors the local space structure dynamic information, in order to understand the physical space.

[0026] Specifically, the plurality of preset category objects include scene background, personnel, object, robot and IOT device, scene behavior event and scene macro rule.

[0027] Specifically, the space structure dynamic information includes dynamic change semantic information of each target object in the target area; it can be understood that: the space intelligence machine performs difference analysis on continuous image frames to obtain the dynamic change of each object.

[0028] The above, by identifying a plurality of target objects from the scene image set, the decomposition of different objects is realized, which is beneficial to the difference analysis of each independent object, thereby improving the difference analysis accuracy, and the obtained space structure dynamic information can master the macro dynamic change of the target area, which is beneficial to the semantic analysis of the dynamic event in the scene.

[0029] S200, for any target object, based on the space structure dynamic information, a preset difference recognition algorithm is used to identify the difference information of the target object in a plurality of dimensions; wherein the difference recognition algorithm corresponding to the scene background, personnel, object, robot and IOT device, scene behavior event and scene macro rule is semantic segmentation and scene recognition algorithm, human pose estimation and behavior recognition algorithm, target detection and tracking algorithm, device state recognition algorithm, event detection algorithm and statistical analysis algorithm.

[0030] Specifically, the difference information is semantic information corresponding to the difference of the target object in any dimension; for example, when detecting the position movement, posture as bending and standing up, and action as picking up the object of the personnel in three dimensions, the corresponding semantic information is moving to the object and bending to pick up the object.

[0031] Specifically, the structural changes in the scene background are identified through a semantic segmentation and scene recognition algorithm; the changes in the position, posture and action of personnel are identified through a human posture estimation and behavior recognition algorithm; the changes in the position, shape, number and spatial position of objects are detected through a target detection and tracking algorithm, such as the movement of tables and chairs; the state, position changes and functional state differences of robots and IOT devices are detected in real time through a device state recognition algorithm; the occurrence or changes of various scene events in the space are identified through an event detection algorithm, such as gathering, abnormal behavior, special events, etc.; the abnormal changes in the macro behavior rules or patterns in the scene are identified and predicted through a statistical analysis algorithm and a machine learning algorithm, such as the trend of changes in the density of people flow, abnormal frequency of device use, etc. Those skilled in the art are aware of the specific implementation of each of the above algorithms, which will not be described here.

[0032] The above-mentioned difference recognition algorithm corresponding to each target object is used to identify the difference information of the target object in multiple dimensions, and different difference recognition algorithms are used for different target objects, so that the difference information of different target objects in multiple dimensions can be accurately identified, the identified difference information is more comprehensive, and the subsequent accurate query is facilitated.

[0033] S300, the difference information of each target object in different dimensions is input into a preset world model, and a target difference feature vector corresponding to a scene image set is output by the preset world model.

[0034] Specifically, the step S300 includes the following steps:

[0035] S301, for any target object, an initial difference feature vector corresponding to the target object is generated by the preset world model according to the difference information of the target object in multiple dimensions; it can be understood that the preset world model extracts the difference information of each target object in different dimensions and uniformly expresses it as a feature vector.

[0036] S302, the initial difference feature vector corresponding to each target object is merged by the preset world model, and a target difference feature vector corresponding to a scene image set is output; it can be understood that the preset world model is embedded in the corresponding space intelligent machine.

[0037] The above-mentioned world model can uniformly quantify the difference information of multiple dimensions, and the world model has the characteristics of efficiently processing high-dimensional space data, so that the extraction of the difference feature vector is more unified and efficient, which is beneficial to real-time and accurate scene difference analysis of the complex scene in the present application.

[0038] S400, the target difference feature vector corresponding to the scene image set and the preset original scene data are associated and stored in the space intelligent machine.

[0039] Specifically, the preset original scene data includes a scene image set corresponding to the target difference feature vector, a reading of a preset sensor, and an original scene description text.

[0040] As described above, due to the acquisition of the difference feature vector, only the original scene data needs to be stored when storing information, and the scene change information is stored in the form of the difference feature vector, instead of storing the global scene data, which effectively avoids repeated storage of a large amount of similar or redundant original scene data, significantly reduces the occupation of the data storage space, and only needs to search and infer the difference information during subsequent scene query, which greatly reduces the data query and processing complexity and improves the efficiency of real-time response and query of the system.

[0041] Further, the method further includes the following steps:

[0042] S10, when receiving a user complex task query request, the user complex task query request is decomposed into a plurality of sub-query tasks by a pre-deployed hub machine; the user complex task query request refers to a request including at least two query tasks; for example, when it is detected that a person enters a target area and approaches a fixed article box, the action and purpose of the person are inferred.

[0043] S20, for each sub-query task, a target difference feature vector corresponding to the sub-query task and original scene data associated with the extracted target difference feature vector are extracted from each space intelligent machine pre-deployed, and semantic information corresponding to the extracted target difference feature vector is obtained.

[0044] In one specific embodiment, the S20 step includes the following steps:

[0045] S21, the semantic information of the sub-query task is obtained and converted into a task feature vector corresponding to the sub-query task; the vector dimension of the task feature vector is consistent with that of the target difference feature vector.

[0046] S22, the vector similarity of the task feature vector corresponding to the sub-query task and each target difference feature vector stored is calculated, and the target difference feature vector corresponding to the maximum vector similarity is extracted.

[0047] In another specific embodiment, a plurality of keywords of the sub-query task and a plurality of semantic keywords corresponding to the target difference feature vector are obtained, and the target difference feature vector corresponding to the most same keywords is extracted by comparing the same keywords.

[0048] Specifically, the space intelligent machine is provided with a plurality of and is distributedly deployed, and each space intelligent machine communicates with the hub machine.

[0049] S30, the semantic information corresponding to the obtained target difference feature vector and the original scene data associated with the extracted target difference feature vector are associated for semantic integration, to obtain a task reasoning result text. For example, when it is detected that a person enters the monitoring space and approaches the fixed article box, the corresponding target difference feature vector is found according to the semantic similarity of the historical data, the action of the person reaching out the arm in front of the fixed article box is obtained, and the action of opening and closing the door of the fixed article box is obtained. Therefore, it is considered that the purpose of the person is to take the articles in the fixed article box.

[0050] The above, through the cooperation of the hub machine and the plurality of space intelligent machines, the hub machine can request the historical target difference feature vector information and the associated original scene data from different space intelligent machines after the complex query task is decomposed, and then the hub machine integrates all the returned data to provide complete and efficient overall reasoning results.

[0051] Embodiment two

[0052] The embodiment two of the present application provides a scene difference information storage system, as shown in the figure, the storage system comprises: Figure 2 As shown in the figure, the storage system comprises:

[0053] The first identification module 100 is used for identifying a plurality of target objects from the scene image set based on a plurality of preset category objects according to the scene image set of the target area collected by the pre-deployed space intelligent machine in a preset time period and the obtained space structure dynamic information; the plurality of preset category objects include scene background, personnel, object, robot and IOT device, scene behavior event and scene macroscopic law;

[0054] The second identification module 200 is used for identifying the difference information of the target object in a plurality of dimensions based on the space structure dynamic information and using a preset difference identification algorithm for the target object; the difference information is the semantic information corresponding to the difference of the target object in any dimension; wherein the difference identification algorithms corresponding to the scene background, personnel, object, robot and IOT device, scene behavior event and scene macroscopic law are semantic segmentation and scene identification algorithm, human body posture estimation and behavior identification algorithm, target detection and tracking algorithm, device state identification algorithm, event detection algorithm and statistical analysis algorithm;

[0055] The processing module 300 is used for inputting the difference information of each target object in different dimensions into a preset world model, and outputting the target difference feature vector corresponding to the scene image set through the preset world model;

[0056] The storage module 400 is used for storing the target difference feature vector corresponding to the scene image set and the preset original scene data into the space intelligent machine.

[0057] It should be noted that the information interaction between the above modules, the execution process and the like, since the same concept based on the method embodiments of the present application, its specific functions and the resulting technical effects, specific can see the method embodiment part, this will not be repeated here.

[0058] While certain embodiments of the application have been described by way of example, it should be appreciated that persons skilled in the art can make modifications to the embodiments without departing from the scope and spirit of the application. The scope of the application is defined by the appended claims.

Claims

1. A method for storing scene difference information, characterized in that, The storage method includes the following steps: S100: Based on the scene image set of the target area collected by the pre-deployed spatial intelligent machine within a preset time period and the obtained spatial structure dynamic information, identify several target objects from the scene image set based on several preset categories of objects; the several preset categories of objects include scene background, people, objects, robots and IoT devices, scene behavioral events and scene macroscopic laws. S200, for any target object, based on the spatial structure dynamic information, a preset difference recognition algorithm is used to identify the difference information of the target object in several dimensions; the difference information is the semantic information corresponding to the difference of the target object in any dimension; wherein, the difference recognition algorithms corresponding to scene background, people, objects, robots and IoT devices, scene behavior events and scene macroscopic laws are respectively semantic segmentation and scene recognition algorithm, human posture estimation and behavior recognition algorithm, target detection and tracking algorithm, device status recognition algorithm, event detection algorithm and statistical analysis algorithm; S300 inputs the difference information of each target object in different dimensions into the preset world model, and outputs the target difference feature vector corresponding to the scene image set through the preset world model; S400, the target difference feature vector corresponding to the scene image set and the preset original scene data are associated and stored in the spatial smart machine; The method further includes the following steps: S10, when a user's complex task query request is received, the user's complex task query request is broken down into several sub-query tasks through a pre-deployed central unit; the user's complex task query request refers to a request that includes at least two query tasks. S20, for each subquery task, extract the target difference feature vector corresponding to the subquery task and the original scene data associated with the extracted target difference feature vector from each pre-deployed spatial intelligent machine, and obtain the semantic information corresponding to the extracted target difference feature vector; step S20 also includes the following steps: S21, obtain the semantic information of the subquery task and convert it into the task feature vector corresponding to the subquery task; the task feature vector and the target difference feature vector have the same vector dimension; S22, calculate the vector similarity between the task feature vector corresponding to the subquery task and each stored target difference feature vector, and extract the target difference feature vector corresponding to the largest vector similarity; S30, semantically integrate the semantic information corresponding to the acquired target difference feature vector with the original scene data associated with the extracted target difference feature vector to obtain the task reasoning result text.

2. The method for storing scene difference information according to claim 1, characterized in that, The preset raw scene data includes a set of scene images corresponding to the target difference feature vector, readings from preset sensors, and raw scene description text.

3. The method for storing scene difference information according to claim 1, characterized in that, The space intelligent machine is configured in several units and is deployed in a distributed manner, with each space intelligent machine communicating with the central machine.

4. The method for storing scene difference information according to claim 1, characterized in that, The spatial structure dynamic information includes the semantic information of the dynamic changes of each target object within the target area.

5. The method for storing scene difference information according to claim 1, characterized in that, The S300 procedure includes the following steps: S301, For any target object, based on the difference information of the target object in several dimensions, generate the initial difference feature vector corresponding to the target object through a preset world model; S302 merges the initial difference feature vectors corresponding to each target object through a preset world model and outputs the target difference feature vectors corresponding to the scene image set.

6. A storage system for scene difference information, characterized in that, The storage system includes: The first recognition module is used to identify several target objects from the scene image set based on several preset categories of objects, according to the scene image set of the target area collected by the pre-deployed spatial intelligent machine within a preset time period and the obtained spatial structure dynamic information; the several preset categories of objects include scene background, people, objects, robots and IoT devices, scene behavior events and scene macroscopic laws. The second identification module is used to identify the difference information of any target object in several dimensions based on the spatial structure dynamic information using a preset difference identification algorithm. The difference information is the semantic information corresponding to the difference of the target object in any dimension. The difference identification algorithms corresponding to the scene background, people, objects, robots and IoT devices, scene behavior events and scene macroscopic laws are semantic segmentation and scene recognition algorithms, human pose estimation and behavior recognition algorithms, target detection and tracking algorithms, device status recognition algorithms, event detection algorithms and statistical analysis algorithms, respectively. The processing module is used to input the difference information of each target object in different dimensions into the preset world model, and output the target difference feature vector corresponding to the scene image set through the preset world model; The storage module is used to associate and store the target difference feature vector corresponding to the scene image set and the preset original scene data into the spatial intelligent machine; The storage system is also used for: When a user's complex task query request is received, the user's complex task query request is broken down into several sub-query tasks through a pre-deployed central unit; the user's complex task query request refers to a request that includes at least two query tasks. For each subquery task, the system extracts the target difference feature vector corresponding to the subquery task and the original scene data associated with the extracted target difference feature vector from each pre-deployed spatial intelligent machine, and obtains the semantic information corresponding to the extracted target difference feature vector. The storage system is also used to: obtain the semantic information of the subquery task and convert it into a task feature vector corresponding to the subquery task; the task feature vector and the target difference feature vector have the same vector dimension; calculate the vector similarity between the task feature vector corresponding to the subquery task and each stored target difference feature vector, and extract the target difference feature vector corresponding to the largest vector similarity. The semantic information corresponding to the acquired target difference feature vector and the original scene data associated with the extracted target difference feature vector are semantically integrated to obtain the task reasoning result text.

Images