An indoor space environment detection method based on a cognitive network

Abstract

The present invention relates to the field of environmental detection, and in particular to a method for indoor space environment detection based on a cognitive network, which solves the problem of rapid detection in indoor space environments. The technical solution adopted by the present invention is: collecting indoor space image data and annotating the image data; constructing a cognitive network model; recognizing the indoor space image based on the cognitive network model to obtain indoor space environment cognition results; and obtaining target recognition results and personnel attribute recognition results in the indoor space environment from the indoor space cognition results. The present invention realizes indoor space environment detection based on an unmanned aerial vehicle platform, can realize full-space detection without blind spots in complex environments, and merges the target detection and recognition model and the personnel attribute recognition model into one model. The model is reasonably designed, can greatly reduce the amount of calculation, improve the speed of obtaining indoor space environment results, and achieve good recognition results.

Description

Technical Field

[0001] The present invention relates to the field of environmental detection, and in particular to a method for detecting indoor space environments based on a cognitive network, which solves the problem of rapid detection in indoor space environments. Background Art

[0002] Drones, with their advantages of low cost, ease of operation, and excellent maneuverability, are widely used in both military and civilian applications. In the military, drones are primarily used for reconnaissance and perception, precision attack, electronic countermeasures, communications support, and training support. In the civilian sector, drones are primarily used for high-definition aerial photography, land surveying and mapping, pesticide spraying, power inspections, geological exploration, forest fire prevention, emergency response, and logistics support. With the further liberalization of national airspace management policies and policy incentives for the drone industry, drone applications are expanding. Therefore, in the field of indoor space detection, drones are the most suitable platform due to the limited space, complex and changing environment, and the greater difficulty of understanding indoor space.

[0003] Indoor space environment detection primarily involves two tasks: identifying indoor objects and identifying people. A common approach involves using object detection and recognition methods to identify indoor objects for task one, and using person attribute recognition methods to identify people for task two. This approach requires executing two models to obtain indoor space environment detection results. However, the computing power of drone onboard cards is limited, and indoor space environment detection tasks are often urgent, requiring rapid results. Consequently, previous indoor space environment detection processing methods have been unable to quickly produce indoor space environment detection results. Summary of the Invention

[0004] The present invention considers that the target detection and recognition model already extracts deep features from images. Based on these features, the target category can be determined. Therefore, these features can also be used to identify the attributes of human targets. This eliminates the need to construct a separate human attribute recognition model for feature extraction and recognition. Therefore, the target detection and recognition model and the human attribute recognition model can be combined into a single model, effectively reducing the computational complexity. Therefore, the present invention proposes a method for indoor space environment detection based on a cognitive network to address the problem that previous indoor space environment detection methods cannot quickly obtain detection results.

[0005] The technical solution adopted in the present invention is:

[0006] A method for detecting indoor space environment based on a cognitive network includes the following steps:

[0007] (1) Collect indoor space image data and annotate the image data;

[0008] (2) Constructing a cognitive network model and loss function, wherein the architecture of the cognitive network model uses the target detection and recognition network architecture as the basic architecture, including YoloV3, YoloV4 or YoloV5. The result output layer adds n dimensions to the prediction confidence, prediction box position and category for predicting personnel attributes, including landmarks and clothing, where n is the number of personnel attributes;

[0009] The loss function of the cognitive network model is:

[0010]

[0011] Among them, S is the grid size, B is the number of anchor points, λ iou Position loss term weight, Indicates whether the overlap ratio between the anchor point at the corresponding position and the standard box is greater than the threshold. If so, the threshold is 1, otherwise the threshold is 0; Indicates whether the overlap ratio between the anchor point at the corresponding position and the standard box is less than the threshold. If so, the threshold is 1, otherwise the threshold is 0;

[0012] in, is the position loss term, λ iou is the weight of the position loss term, IOU represents the intersection-union ratio between the predicted box and the real box, ρ 2 (b,b gt ) is the square of the Euclidean distance between the center point of the predicted box and the real box, where b gt and b represent the center coordinates of the real box and the predicted box respectively, and c is the diagonal distance of the minimum rectangular area that can contain the predicted box and the standard box. is the difference between the predicted aspect ratio and the standard aspect ratio, where w gt and h gt Represent the width and height of the real box, w and h represent the width and height of the predicted box, and α is a parameter;

[0013] in, is the target confidence loss, For targeted losses, is the loss without target, λ cls is the target confidence loss, λ c is the category loss, is the true value of the target at the corresponding position, C i is the predicted value of the target at the corresponding position, Indicates whether the overlap ratio between the anchor point at the corresponding position and the standard box is less than the threshold. If so, the threshold is 1, otherwise the threshold is 0;

[0014] in, is the target category loss function, classes is the target category type at the corresponding position, is the true probability of the target, p i (c) is the predicted probability of the target;

[0015] in Identify loss items for person attributes, Indicates whether the corresponding position contains a human target. 1 indicates that it contains a human target, and 0 indicates that it does not contain a human target, that is: The target is the output result of the target recognition result. The attribute represents the attribute of the person. represents the true value of the person's attribute, q i (c) represents the predicted value of personnel attributes;

[0016] (3) Based on the cognitive network model, the indoor space image is recognized to obtain the indoor space environment cognitive results. The cognitive results are expressed as follows:

[0017] For an image X, a tap of the model outputs a tensor of k×k×(5+classes+att)×B, where k×k is the location network, 5 represents an object confidence and the four dimensions x, y, w, h representing the location, classes is the number of detection categories, att is the number of person attributes, and B is the number of anchor points. That is, the output vector of a certain location is [c,x,y,w,h,0,1,0,…,1,0,0,1,1,…]×B; the object confidence is whether there is an object at the corresponding location;

[0018] (4) Obtain target recognition results and personnel attribute recognition results in the indoor space environment from the indoor space cognition results.

[0019] Furthermore, in step (1), the image data is labeled as follows:

[0020] For an image X, the corresponding object is labeled y = [x, y, w, h, 0, 1, 0, ...], where x, y, w, h represent location information and 0, 1, 0, ... represent category information;

[0021] If the target is a person target, it corresponds to a set of attributes, and each attribute is represented by 1 or 0 to indicate presence or absence. The notation format is y = [x, y, w, h, 0, 1, 0, …, 1, 0, 0, 1, 1, …], where the last n digits represent the person attribute information, and n is the number of person attributes, including markers and clothing.

[0022] Furthermore, step (4) is specifically as follows:

[0023] From the indoor space cognition results, the target detection part is taken to obtain the target detection result. If the detected target is a person, the person attribute recognition part result is taken to obtain the person attribute recognition result.

[0024] Compared with the prior art, the present invention has the following beneficial effects:

[0025] 1. The present invention realizes indoor space environment detection based on the UAV platform, and can achieve full-space detection without blind spots in complex environments;

[0026] 2. The present invention combines the target detection and recognition model and the person attribute recognition model into one model, which can greatly reduce the amount of calculation and improve the speed of obtaining indoor space environment results;

[0027] 3. The present invention applies the feature extraction results of the target detection and recognition model to personnel attribute recognition, with reasonable design and good recognition results. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 This is a flow chart of the indoor space environment detection method provided by an embodiment of the present invention.

[0029] Figure 2 This is a schematic diagram of image annotation provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0030] The following will be combined with the embodiments to clearly and completely describe the concept, technical solution advantages and technical effects of the present invention so as to fully understand the purpose, features and effects of the present invention. It should be noted that the specific embodiments described herein are only used to explain the present invention and do not limit the present invention.

[0031] See Figure 1 This embodiment provides a method for detecting an indoor space environment based on a cognitive network, the method comprising:

[0032] S1: Collect indoor space image data and annotate the image data:

[0033] For an image X, the label of an object is y = [x, y, w, h, 0, 1, 0, ...], that is, the position information and category information are in the form of one-hot vectors; where x, y, w, h represent the position information, and 0, 1, 0, ... represent the category information;

[0034] If the target is a person, it corresponds to a set of attributes, each attribute is represented by 1 or 0, and the notation format is y = [x, y, w, h, 0, 1, 0, ..., 1, 0, 0, 1, 1, ...], where the last n digits represent the person's attribute information, and n is the number of person attributes, including markers and clothing. Figure 2, where the 1 or 0 in the second half represents the personnel attributes;

[0035] The target categories of the indoor space environment of the present invention include: knives, guns, ammunition, furniture, pipes, important signs, doors, windows, switches, fire extinguishers, lights, personnel, and vehicles; personnel attributes include: weapons (guns, knives), clothing (camouflage, casual clothes).

[0036] S2: Construct a cognitive network model and loss function. The architecture of the cognitive network model uses the target detection and recognition network architecture as the basic architecture, including YoloV3, YoloV4 or YoloV5. The result output layer adds n dimensions to predict personnel attributes, including landmarks and clothing, based on the prediction confidence, prediction box position and category. n is the number of personnel attributes.

[0037] The loss function is specifically:

[0038]

[0039] Among them, S is the grid size, B is the number of anchor points, λ iou Position loss term weight, Indicates whether the overlap ratio between the anchor point at the corresponding position and the standard box is greater than the threshold. If so, the threshold is 1, otherwise the threshold is 0; Indicates whether the overlap ratio between the anchor point at the corresponding position and the standard box is less than the threshold. If so, the threshold is 1, otherwise the threshold is 0;

[0040] in, is the position loss term, λ iou is the weight of the position loss term, IOU represents the intersection-union ratio between the predicted box and the real box, ρ 2 (b,b gt ) is the square of the Euclidean distance between the center point of the predicted box and the real box, where b gt and b represent the center coordinates of the real box and the predicted box respectively, and c is the diagonal distance of the minimum rectangular area that can contain the predicted box and the standard box. is the difference between the predicted aspect ratio and the standard aspect ratio, where w gt and h gt Represent the width and height of the real box, w and h represent the width and height of the predicted box, and α is a parameter;

[0041] in, is the target confidence loss, For targeted losses, is the loss without target, λ cls is the target confidence loss, λ c is the category loss, is the true value of the target at the corresponding position, C i is the predicted value of the target at the corresponding position, Indicates whether the overlap ratio between the anchor point at the corresponding position and the standard box is less than the threshold. If so, the threshold is 1, otherwise the threshold is 0;

[0042] in, is the target category loss function, classes is the target category type at the corresponding position, is the true probability of the target, p i (c) is the predicted probability of the target;

[0043] in Identify loss items for person attributes, Indicates whether the corresponding position contains a human target. 1 indicates that it contains a human target, and 0 indicates that it does not contain a human target. Attribute indicates the human attribute. represents the true value of the person's attribute, q i (c) represents the predicted value of personnel attributes.

[0044] S3: Based on the cognitive network model, the indoor space image is recognized to obtain the indoor space environment recognition results:

[0045] The indoor space environment cognition results can be expressed as:

[0046] For an image X, a tap of the model outputs a tensor of k×k×(5+classes+att)×B, where k×k is the location network, 5 represents an object confidence (i.e., whether there is an object at that location, but it does not matter what kind of object it is) and the four dimensions x, y, w, h representing the location, classes is the number of detection categories, att is the number of person attributes, and B is the number of anchor points. That is, the output vector for a certain location is [c,x,y,w,h,0,1,0,…,1,0,0,1,1,…]×B.

[0047] S4: Obtain target recognition results and person attribute recognition results in the indoor space environment from the indoor space cognition results:

[0048] From the indoor space cognition results, the target detection part is taken to obtain the target detection result. If the detected target is a person, the person attribute recognition part result is taken to obtain the person attribute recognition result.

Claims

1. A method for detecting indoor space environment based on cognitive network, characterized in that: The following steps are involved: (1) Collect indoor space image data and annotate the image data; (2) Constructing a cognitive network model and loss function, wherein the architecture of the cognitive network model uses the target detection and recognition network architecture as the basic architecture, including YoloV3, YoloV4 or YoloV5. The result output layer adds n dimensions to the prediction confidence, prediction box position and category for predicting personnel attributes, including landmarks and clothing, where n is the number of personnel attributes; The loss function of the cognitive network model is: Among them, S is the grid size, B is the number of anchor points, λ iou Position loss term weight, Indicates whether the overlap ratio between the anchor point at the corresponding position and the standard box is greater than the threshold. If so, the threshold is 1, otherwise the threshold is 0; Indicates whether the overlap ratio between the anchor point at the corresponding position and the standard box is less than the threshold. If so, the threshold is 1, otherwise the threshold is 0; in, is the position loss term, λ iou is the weight of the position loss term, IOU represents the intersection-union ratio between the predicted box and the real box, ρ 2 (b,b gt ) is the square of the Euclidean distance between the center point of the predicted box and the real box, where b gt and b represent the center coordinates of the real box and the predicted box respectively, and c is the diagonal distance of the minimum rectangular area that can contain the predicted box and the standard box. is the difference between the predicted aspect ratio and the standard aspect ratio, where w gt and h gt Represent the width and height of the real box, w and h represent the width and height of the predicted box, and α is a parameter; in, is the target confidence loss, For targeted losses, is the loss without target, λ cls is the target confidence loss, λ c is the category loss, is the true value of the target at the corresponding position, C i is the predicted value of the target at the corresponding position, Indicates whether the overlap ratio between the anchor point at the corresponding position and the standard box is less than the threshold. If so, the threshold is 1, otherwise the threshold is 0; in, is the target category loss function, classes is the target category type at the corresponding position, is the true probability of the target, p i (c) is the predicted probability of the target; in Identify loss items for person attributes, Indicates whether the corresponding position contains a human target. 1 indicates that it contains a human target, and 0 indicates that it does not contain a human target. Attribute indicates the human attribute. represents the true value of the person's attribute, q i (c) represents the predicted value of personnel attributes; (3) Based on the cognitive network model, the indoor space image is recognized to obtain the indoor space environment recognition result, which is expressed as: For an image X, a tap of the model outputs a tensor of k×k×(5+classes+att)×B, where k×k is the location network, 5 represents an object confidence and the four dimensions x, y, w, h representing the location, classes is the number of detection categories, att is the number of person attributes, and B is the number of anchor points. That is, the output vector of a certain location is [c,x,y,w,h,0,1,0,…,1,0,0,1,1,…]×B; the object confidence is whether there is an object at the corresponding location; (4) Obtain target recognition results and personnel attribute recognition results in the indoor space environment from the indoor space cognition results.

2. The method for detecting indoor space environment based on cognitive network according to claim 1, characterized in that: In step (1), the image data is labeled as follows: For an image X, the corresponding object is labeled y = [x, y, w, h, 0, 1, 0, ...], where x, y, w, h represent location information and 0, 1, 0, ... represent category information; If the target is a person target, it corresponds to a set of attributes, and each attribute is represented by 1 or 0 to indicate presence or absence. The notation format is y = [x, y, w, h, 0, 1, 0, …, 1, 0, 0, 1, 1, …], where the last n digits represent the person attribute information, and n is the number of person attributes, including markers and clothing.

3. The method for detecting indoor space environment based on cognitive network according to claim 1, characterized in that: Step (4) is specifically as follows: From the indoor space cognition results, the target detection part is taken to obtain the target detection result. If the detected target is a person, the person attribute recognition part result is taken to obtain the person attribute recognition result.

4. The method for detecting indoor space environment based on cognitive network according to claim 2, characterized in that: Indoor space environment target categories include: knives, guns, ammunition, furniture, pipes, important signs, doors, windows, switches, fire extinguishers, lights, people and vehicles; personnel attributes include: weapons and clothing.

Images