A general viewpoint planning method based on knowledge base reasoning

Through a general viewpoint planning method based on knowledge base reasoning, combined with conditional entropy and weighted information gain, automated viewpoint planning for different visual tasks is achieved, and the problem that existing methods cannot be applied to target modeling and target recognition tasks at the same time is solved, improving Task efficiency and initiative.

CN114429234BActive Publication Date: 2024-11-22SHENYANG INST OF AUTOMATION - CHINESE ACAD OF SCI
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202011170971.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-10-28
Publication Date
2024-11-22
Estimated Expiration
2040-10-28

AI Technical Summary

Technical Problem

The existing viewpoint planning method cannot be applied to both goal modeling tasks and goal recognition tasks, and there is a lack of a general viewpoint planning method.

Method used

A general viewpoint planning method based on knowledge base reasoning is adopted to automatically calculate the pose of the next best viewpoint through formal description of visual tasks and prior information, real-time state update and unknown area prediction, combined with conditional entropy and weighted information gain.

Benefits of technology

It realizes automated viewpoint planning for different visual tasks, improves task efficiency and initiative, makes full use of the prior information of the measured target, and is suitable for a variety of visual task scenarios.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114429234B_ABST
    Figure CN114429234B_ABST
Patent Text Reader

Abstract

The present invention relates to the field of active perception, and more specifically, to a general viewpoint planning method based on knowledge base reasoning. The general viewpoint planning method based on knowledge base reasoning includes: a description method of visual tasks and prior information, which formally expresses the task state of the visual task, the candidate state of the object to be measured, the perception state and the prior information, and inputs the active perception system according to these formal expressions; an update method for each state and a prediction method for the unknown area of ​​the model space, which updates each state in real time according to the detection information, and predicts the unknown area of ​​the target using the object prior information; a next viewpoint determination method based on the formal description, which uses the conditional entropy to measure the weight of each feature, and uses a viewpoint evaluation method of weighted information gain to determine the next best viewpoint. The present invention can be applied to target modeling and target recognition tasks, so that the viewpoint planning method is separated from the specific visual task and has a wider range of applications.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of active perception, and in particular to a general viewpoint planning method based on knowledge base reasoning. Background Art

[0002] In the process of performing visual tasks, the data obtained from a single viewpoint may not be sufficient to complete the task objectives, so it is necessary to detect the target from multiple viewpoints. Active perception is to improve the efficiency of visual tasks and reduce the energy consumption of the sensor-attached platform by purposefully adjusting the position and posture of the visual sensor. The selection of the next best viewpoint is the core goal of the active perception task, which determines the contribution of the perception data obtained from the next viewpoint to the task objectives. At present, there are some viewpoint planning methods for specific tasks.

[0003] For target recognition tasks, there are mainly morphological graph-based methods and deep learning-based methods. The former uses morphological graphs to represent targets, uses the target morphology extracted from two-dimensional views at different viewing angles as target features, describes the positional dependency between features in a graph-based manner, and uses the Bayesian method to calculate the distinguishability of different features, thereby determining the next best viewpoint. The latter uses an end-to-end approach, using the sensor's detection image at the current viewpoint as the network input, and the trained network can directly output the next best viewpoint.

[0004] For target modeling tasks, there are mainly methods based on surface prediction and methods based on information gain. The former predicts the unknown surface based on the known surface of the target and the surface ductility of smooth objects, and then determines the next best viewpoint based on the predicted surface. The latter uses a voxel model to assign a larger information gain to voxels that are of great value for information acquisition, such as unknown voxels and voxels close to the surface, and then performs statistics on the voxel information gain of candidate viewpoints in the voxel space, and selects the viewpoint with the larger information gain as the next best viewpoint.

[0005] However, the application task scenarios of the above methods all have certain limitations. At present, there is no universal viewpoint planning method that is suitable for both viewpoint planning in target modeling tasks and next viewpoint determination in target recognition tasks. Summary of the invention

[0006] In order to overcome the limitations of existing methods, the present invention proposes a general viewpoint planning method based on partial prior information of objects which is applicable to different scenes.

[0007] The technical solution adopted by the present invention to achieve the above-mentioned purpose is:

[0008] A general viewpoint planning method based on knowledge base reasoning includes: a description method of visual tasks and prior information, a real-time update method of each state and a prediction method of unknown areas, and a general viewpoint planning method based on formal description, wherein:

[0009] Description method of visual tasks and prior information: formally express the task state of the visual task, the candidate state space of the object being measured, the perception state and prior information, and use the unified formal expression of different visual tasks as the input of the viewpoint planning method;

[0010] Real-time update method of each state and prediction method of unknown features: guide the real-time update of candidate states and perception states of the object under test during the actual detection process: based on the perception data including point cloud and image of the object under test obtained by the visual sensor, the voxel model space where the object under test is located is updated in real time. With the migration of candidate states and perception states of the object under test, the visual task gradually approaches its target state. When the target state is not reached, the unknown area of ​​the model space is predicted based on the current perception state of the object under test and its prior information library;

[0011] A general viewpoint planning method based on formal description: predict the next best viewpoint of the visual sensor: first use conditional entropy to measure the distinctiveness of each feature to express the corresponding weight of the viewpoint feature voxel gain; then use the viewpoint evaluation method of weighted information gain to calculate the evaluation function value of the candidate viewpoint, and select the viewpoint with the largest evaluation function value as the next best viewpoint.

[0012] The detected object's perception state perc_s t The description method is:

[0013] perc_s t :(f t1 ,f t2 ..f tn ...f tN )

[0014] f tn is the perception state of feature n at time t, f n It is the nth feature that defines the object being detected. The visual task contains a total of N features to be detected.

[0015] The candidate state space candi_s of the object under test is a possible combination of each characteristic state, and its description method is as follows:

[0016] candi_s:{O1,...O j ,...O J}

[0017] Among them, O jRefers to the characteristic states of the jth possible type of the object being measured, and the description method is as follows:

[0018] O j :(f j1 ,f j2 ,...f jn ...f jN )

[0019] Among them, f jn It represents the corresponding feature state of feature n in the measured object j. If the feature exists, it is set to 1, otherwise it is set to 0. The feature state space includes -1, 0, 2, and 1, where -1 represents the initial state of the feature and no relevant information of the feature has been detected; 0 represents that the feature does not exist; 1 represents that the feature exists and has been detected; and 2 represents that the feature exists but has not been detected.

[0020] The task status task_s t The description method is (cfd t ,mapIg t ), cfd t It represents the confidence of the target at time t, and the calculation formula is as follows:

[0021] cfd t =1 / J

[0022] Among them, J refers to the number of candidate states in the candidate state space, mapIg t It represents the information entropy state of the voxel model space at time t, and the calculation formula is as follows:

[0023]

[0024] Among them, p i represents the occupancy probability of voxel i.

[0025] The feature is that the description of the prior information is in the form of rules, and each rule k It includes the feature verification method, feature attribute calculation method, and feature derivation method, in the following form:

[0026] rule k :{(an1,...an m ) →cons k ,test k (),cal k (),pre k ()}

[0027] The rule uses features an1~an m Derived feature cons k Related information, test k() is used to verify the feature cons k Can it be verified to exist? k () Calculate the feature cons k The properties of pre k () is based on the known features an1~an m Prediction featurescons k Function corresponding to the point cloud.

[0028] The perceived state of the object being measured perc_s t The real-time update method of each feature’s perception state f tn The update is performed by the detection information of the visual sensor or the derivation of the candidate state space. The update method is as follows:

[0029]

[0030] num_crt n Refers to the characteristic f n The number of existing predicted voxels, num_tt n Refers to the characteristic f n The predicted point cloud corresponds to the total number of voxels, and thresh is a manually set threshold.

[0031] The real-time updating method of the candidate state space, when the perception state of the feature is updated according to the information of the voxel model space, the candidate state space is updated according to the update of the feature, and the conditions for screening the candidate state space are as follows:

[0032]

[0033] The prediction method for unknown areas is to select rules for predicting unknown features according to feature states, and use the function predict described in the rules to predict unknown features. k (), derive the point cloud distribution of unknown features, set the unknown voxels corresponding to the point cloud whose distance from the known voxels exceeds the threshold as predicted voxels, and mark their feature categories and feature sub-region numbers. The feature sub-region represents one of the regions that can derive the complete attributes of the feature, that is, a complete description of the feature.

[0034] The corresponding weight of the viewpoint feature voxel gain, that is, the distinguishability of each feature, is measured using conditional entropy. The conditional entropy formula is as follows

[0035]

[0036] Among them, p(o j ,f n ) refers to the characteristic state f n , the object to be measured is o j The joint probability, p(o j |fn ) refers to when the characteristic state is f n When the target belongs to the conditional probability of the jth type of object being measured, for the feature with a perception state of -1, the weight w i Normalization calculation is performed based on conditional entropy; for features with state 2, the weight is 0.01, which is the minimum normalized weight.

[0037] The candidate viewpoint evaluation method uses the expression of weighted information gain, and the evaluation function G(v) is as follows:

[0038]

[0039] Among them, g_overlap v Refers to the overlapping information gain of viewpoint v. When the overlapping ratio of the current and subsequent frames is greater than the set threshold, the information gain is set to 1, otherwise, this item is set to 0; W1 and W2 are the weights of the predicted feature voxel information gain and the weights of the unknown voxels, respectively. It refers to the ratio of the number of feature i voxels detectable from the current viewpoint to the total number of feature i predictions. Refers to the ratio of the number of unknown voxels at the current viewpoint to the total number of unknown voxels. The present invention has the following advantages and beneficial effects:

[0040] 1. The present invention makes full use of the prior information of the target to be measured;

[0041] 2. The present invention is applicable to different visual tasks;

[0042] 3. After the present invention formally describes the visual task, the algorithm can automatically calculate the position and posture of the next viewpoint without human intervention, thereby improving the initiative and efficiency of the visual task. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a block diagram of the general viewpoint planning algorithm of the present invention. DETAILED DESCRIPTION

[0044] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments:

[0045] A general viewpoint planning method is proposed for various visual tasks such as 3D target modeling or 3D target recognition. The next best viewpoint of the sensor is determined based on the detected data and known information of the target. For example, when the visual task is target recognition, the target state of the task is (1,0). If the target belongs to one of the two types of objects, these objects can be described by three features, namely, whether there is a circle at 1, whether there is a triangle at 2, and whether there is a square at 3. The feature state in category 1 is (1,1,1), and the feature state in category 2 is (1,0,1). The candidate state space of the object is represented as (1,1,1) and (1,0,1); initially, if all features are not detected, the initial perception state is (-1,-1,-1). The verification method of the features stored in the prior information library and the inference prediction method between the features are used. The following process is repeated until the task reaches the target state: the visual sensor moves to the specified viewpoint, obtains the image or point cloud data of the target, and after updating the voxel model space with the original data, verifies the existence of each feature according to the rules in the prior information library. Assuming that feature 1 is verified to exist, and features 2 and 3 cannot be verified, the current perception state is updated to (1, -1, -1). According to the relevant rules in the prior information library, predict the point clouds of 2 and 3, mark the corresponding predicted voxels in the voxel model space, and calculate the weights corresponding to features 2 and 3. Finally, use the viewpoint evaluation function to evaluate the contribution of each candidate viewpoint to the visual task, and select the viewpoint with the largest contribution as the next best viewpoint.

[0046] The scheme adopted by the present invention includes: a formal description method of visual tasks and prior information; an update method of each state and a prediction method of unknown features; and a next viewpoint determination method based on the formal description.

[0047] The description method of visual tasks and prior information formally expresses the task state of the visual task, the candidate state space of the measured object, the perception state and prior information, and inputs different visual tasks into the viewpoint planning system through a unified formal expression.

[0048] The real-time update method of each state and the prediction method of unknown features are used to guide the update of the candidate state and perception state of the object under test during the actual detection process. With the migration of each state, the visual task gradually approaches its target state. When the target state is not reached, the unknown area of ​​the model space is predicted based on the current perception state of the object under test and its prior information library.

[0049] A general viewpoint planning method based on formal description is used to predict the next best viewpoint of the sensor. First, conditional entropy is used to measure the distinguishability of each feature to express the corresponding weight of the viewpoint feature voxel gain. Then a weighted information gain viewpoint evaluation method is used to calculate the evaluation function value of the candidate viewpoint and select the next best viewpoint.

[0050] Among them, feature status refers to the status of each feature;

[0051] The candidate state refers to the possible category of objects to which the object being tested belongs. In each object, the state of each feature is different, and the combination of each feature state constitutes the candidate state of the object;

[0052] Task state refers to the current state of the visual task;

[0053] The perception status refers to the real-time status of each feature analyzed based on the detection data.

[0054] The detected object's perception state perc_s t The description method is:

[0055] perc_s t :(f t1 ,f t2 ..f tn ...f tN ) (1)

[0056] f tn refers to the state of feature n at time t, f n It is the nth feature that defines the object to be detected. The visual task contains N features to be detected. The state space of the feature includes -1, 0, 2, and 1. -1 indicates the initial state of the feature, and no relevant information of the feature has been detected; 0 indicates that the feature does not exist; 1 indicates that the feature exists and has been detected; 2 indicates that the feature exists but has not been detected.

[0057] The candidate state space candi_s of the object under test is a possible combination of each characteristic state, and its description method is as follows:

[0058] candi_s:{O1,...O j ,...O J} (2)

[0059] Among them, O j Refers to the characteristic states of the jth possible type of the detected object, and the description method is as follows:

[0060] O j :(f j1 ,f j2 ,...f jn ...f jN ) (3)

[0061] In the above formula, f jn Indicates the corresponding state of feature n in target j. If the feature exists, it is set to 1, otherwise it is set to 0.

[0062] The task status task_s tThe description method is (cfd t ,mapIg t ), cfd t It represents the confidence of the target at time t, and the calculation formula is as follows:

[0063] cfd t =1 / J (4)

[0064] Among them, J refers to the number of states in the candidate state set, mapIg t It represents the information entropy state of the voxel model space at time t, and the calculation formula is as follows:

[0065]

[0066] Among them, p i represents the occupancy probability of voxel i.

[0067] The description of the prior information is in the form of rules, each rule k It includes the feature verification method, feature attribute calculation method, and feature derivation method, in the following form:

[0068] rule k :{(an1,...an m ) →cons k ,test k (),cal k (),pre k ()} (6)

[0069] This rule uses features an1~an m Derived feature cons k Related information, test k () is used to verify the feature cons k Can it be verified to exist? k () Calculate the feature cons k The properties of pre k () is the prediction feature cons k Function corresponding to the point cloud.

[0070] The perceived state perc_s of the object being measured t Real-time update method, the state of each feature bit f tn The update is performed by the detection information or the derivation of the candidate state set. The update method is as follows:

[0071]

[0072] num_crt n Refers to the characteristic f nThe number of existing predicted voxels, num_tt n Refers to the characteristic f n The predicted point cloud corresponds to the total number of voxels, and thresh is a manually set threshold.

[0073] In the real-time updating method of the candidate state set, when the state of the feature is updated according to the information of the model space, the candidate state set is updated according to the update of the feature bit, and the conditions for screening the candidate state set are as follows:

[0074]

[0075] The prediction method of the unknown region selects rules that can be used to predict unknown features according to the feature state, and the function predict described in the rule is used to predict the feature. k (), derive the point cloud distribution of unknown features, set the unknown voxels corresponding to the point cloud whose distance from the known voxels exceeds the threshold as predicted voxels, and mark their feature categories and feature sub-region numbers. The feature sub-region represents one of the regions from which the complete attributes of the feature can be derived.

[0076] The corresponding weight of the viewpoint feature voxel gain, that is, the distinguishability of each feature, is measured using conditional entropy. The conditional entropy formula is as follows

[0077]

[0078] For features with a perceptual state of -1, the weight w i Normalization is performed based on conditional entropy. For features with state 2, the weight is 0.01, which is the minimum normalized weight.

[0079] The candidate viewpoint evaluation method uses the expression of weighted information gain, and the formula is as follows.

[0080]

[0081] Among them, g_overlap v Refers to the overlapping information gain of viewpoint v. When the overlapping ratio of the current and subsequent frames is greater than the set threshold, the information gain is set to 1, otherwise, this item is set to 0. W1 and W2 are the weights of the predicted feature voxel information gain and the weights of the unknown voxels, respectively. It refers to the ratio of the number of feature i voxels detectable from the current viewpoint to the total number of feature i predictions. Refers to the ratio of the unknown voxels of the current viewpoint to the total unknown voxels. Among them, the predicted feature voxel refers to the unknown voxel predicted to have point clouds distributed.

[0082] according to Figure 1As shown in the figure, the general viewpoint planning algorithm mainly includes four stages: input of the viewpoint planning system, update of the perception state and candidate state, prediction of the unknown area and selection of the next viewpoint.

[0083] In the input stage of the viewpoint planning module, according to the task state of the visual task proposed in the present invention, the candidate state of the measured object, the perception state and the formal expression method of the prior information, the initial state or target state and the prior information of the current visual task related state are input into the viewpoint planning system in a specific form as the basis for the operation of the entire viewpoint planning system.

[0084] In the stage of updating the perception state and candidate state, the perception state of the task is updated according to the perception and prediction information of the model space according to the feature state updating method proposed by the present invention. When the state of the feature changes according to the information of the model space, the candidate state set is updated according to the candidate state updating method proposed by the present invention. After the candidate state set is updated, the perception state of the task is updated again according to the feature corresponding state of the candidate state set.

[0085] In the prediction stage of unknown areas, when a feature is uncertain and unpredicted, the prior information of the visual task is used to deduce its corresponding point cloud distribution from the known features related to it. According to the state update method of unknown voxels proposed by the present invention, the predicted point cloud is used to update the unknown voxel state in the voxel model.

[0086] In the next viewpoint selection stage, according to the viewpoint evaluation method of weighted information gain proposed in the present invention, the distinguishability of each feature is first described by the corresponding conditional entropy. Then, for each candidate viewpoint, according to the unknown feature information gain statistical method proposed in the present invention, the visible ratio of each unknown feature is calculated, and the features are weighted to obtain the maximum weighted visible ratio value of the viewpoint as the evaluation function value of the viewpoint, and finally the viewpoint with the largest evaluation function value is selected as the next best viewpoint.

Claims

1. A general viewpoint planning method based on knowledge base reasoning, characterized in that: include: The description method of visual tasks and prior information, the real-time update method of each state and the prediction method of unknown areas, and the general viewpoint planning method based on formal description, among which: Description method of visual tasks and prior information: formally express the task state of the visual task, the candidate state space of the object being measured, the perception state and prior information, and use the unified formal expression of different visual tasks as the input of the viewpoint planning method; Real-time update method of each state and prediction method of unknown features: guide the real-time update of candidate states and perception states of the object under test during the actual detection process: based on the perception data including point cloud and image of the object under test obtained by the visual sensor, the voxel model space where the object under test is located is updated in real time. With the migration of candidate states and perception states of the object under test, the visual task gradually approaches its target state. When the target state is not reached, the unknown area of ​​the model space is predicted based on the current perception state of the object under test and its prior information library; A general viewpoint planning method based on formal description: predicting the next best viewpoint of the visual sensor: first, conditional entropy is used to measure the distinguishability of each feature to express the corresponding weight of the viewpoint feature voxel gain; then the viewpoint evaluation method of weighted information gain is used to calculate the evaluation function value of the candidate viewpoint, and the viewpoint with the largest evaluation function value is selected as the next best viewpoint; The detected object's perception state perc_s t The description method is: perc_s t :(f t1 ,f t2 ..f tn ...f tN ) f tn is the perception state of feature n at time t, f n It is the nth feature that defines the object to be detected. The visual task contains N features to be detected. The candidate state space candi_s of the object under test is a possible combination of each characteristic state, and its description method is as follows: candi_s:{O1,...O j ,...THE J } Among them, O j Refers to the characteristic states of the jth possible type of the object being measured, and the description method is as follows: O j :(f j1 ,f j2 ,...f jn …f jN ) Among them, f jn Indicates the corresponding feature state of feature n in the object j being measured. If the feature exists, it is set to 1, otherwise it is set to 0. The feature state space includes -1, 0, 2, and 1, where -1 indicates the initial state of the feature and no relevant information of the feature has been detected; 0 indicates that the feature does not exist; 1 indicates that the feature exists and has been detected; 2 indicates that the feature exists but has not been detected; The task status task_s t The description method is (cfd t ,mapIg t ), cfd t It represents the confidence of the target at time t, and the calculation formula is as follows: CFD t =1 / J Among them, J refers to the number of candidate states in the candidate state space, mapIg t It represents the information entropy state of the voxel model space at time t, and the calculation formula is as follows: Among them, p i represents the occupancy probability of voxel i.

2. A general viewpoint planning method based on knowledge base reasoning according to claim 1, characterized in that: The description of the prior information is in the form of rules, each rule k It includes the feature verification method, feature attribute calculation method, and feature derivation method, in the following form: rule k :{(an1,…an m )→cons k ,test k (),cal k (),pre k ()} The rule uses features an1~an m Derived feature cons k Related information, test k () is used to verify the feature cons k Can it be verified to exist? k () Calculate the feature cons k The properties of pre k () is based on the known features an1~an m Prediction featurescons k Function corresponding to the point cloud.

3. A general viewpoint planning method based on knowledge base reasoning according to claim 1, characterized in that: The perceived state of the object being measured perc_s t The real-time update method of each feature’s perception state f tn The update is performed by the detection information of the visual sensor or the derivation of the candidate state space. The update method is as follows: num_crt n Refers to the characteristic f n The number of existing predicted voxels, num_tt n Refers to the characteristic f n The predicted point cloud corresponds to the total number of voxels, and thresh is a manually set threshold.

4. A general viewpoint planning method based on knowledge base reasoning according to claim 1, characterized in that: The real-time updating method of the candidate state space, when the perception state of the feature is updated according to the information of the voxel model space, the candidate state space is updated according to the update of the feature, and the conditions for screening the candidate state space are as follows:

5. A general viewpoint planning method based on knowledge base reasoning according to claim 1, characterized in that: The prediction method for unknown areas is to select rules for predicting unknown features according to feature states, and use the function predict described in the rules to predict unknown features. k (), derive the point cloud distribution of unknown features, set the unknown voxels corresponding to the point cloud whose distance from the known voxels exceeds the threshold as predicted voxels, and mark their feature categories and feature sub-region numbers. The feature sub-region represents one of the regions that can derive the complete attributes of the feature, that is, a complete description of the feature.

6. A general viewpoint planning method based on knowledge base reasoning according to claim 1, characterized in that: The corresponding weight of the viewpoint feature voxel gain, that is, the distinguishability of each feature, is measured using conditional entropy. The conditional entropy formula is as follows Among them, p(o j ,f n ) refers to the characteristic state f n , the object to be measured is o j The joint probability, p(o j |f n ) refers to when the characteristic state is f n When the target belongs to the conditional probability of the jth type of object being measured, for the feature with a perception state of -1, the weight w i Normalization calculation is performed based on conditional entropy; for features with state 2, the weight is 0.01, which is the minimum normalized weight.

7. A general viewpoint planning method based on knowledge base reasoning according to claim 1, characterized in that: The candidate viewpoint evaluation method uses the expression of weighted information gain, and the evaluation function G(v) is as follows: Among them, g_overlap v Refers to the overlapping information gain of viewpoint v. When the overlapping ratio of the current and subsequent frames is greater than the set threshold, the information gain is set to 1, otherwise, this item is set to 0; W1 and W2 are the weights of the predicted feature voxel information gain and the weights of the unknown voxels, respectively. It refers to the ratio of the number of feature i voxels detectable from the current viewpoint to the total number of feature i predictions. Refers to the ratio of the number of unknown voxels at the current viewpoint to all unknown voxels.