A perimeter protection method and device, a terminal and a readable storage medium

Abstract

The application provides a perimeter defense method and device, a terminal and a readable storage medium, and belongs to the technical field of communication, wherein the perimeter defense method comprises the following steps: training a target segmentation network based on a public data set of perimeter defense; constructing a category database according to a live picture of a target site and the target segmentation network; extracting a target feature vector of a target video frame picture by using the target segmentation network; searching in the category database according to the target feature vector to determine whether a target object exists; and determining whether the target object invades the target site according to pixel information of the target object and a warning contour line. In the application, the target segmentation network is trained by using the public data set, and the category database is constructed, so that the dependence on a training data set is reduced, a user can freely construct the category database according to a specific scene, the perimeter defense method of the application can be adapted to various scenes, the searching speed is accelerated, and the perimeter invasion determination is more accurate.

Description

Technical Field

[0001] This invention relates to the field of construction site monitoring technology, and in particular to a perimeter security method and device, terminal and readable storage medium. Background Technology

[0002] Perimeter intrusion prevention utilizes technology and equipment to effectively detect and alarm against illegal intrusion, theft, vandalism, and violations of regulations within a designated perimeter. This reduces unnecessary injuries and fatalities caused by falling objects during overlapping work operations due to not wearing safety helmets, or by unauthorized personnel entering areas prone to collapse during work stoppages.

[0003] However, current perimeter intrusion prevention solutions that use temperature, sound, and light sensors for monitoring are highly limited in application scenarios and cannot distinguish the types of targets to be detected. If cameras are used for monitoring, a large amount of data training is required to achieve stable identification of a certain type of target. When it is necessary to apply the solution to new scenarios and detect new target types, the transfer learning is difficult and costly. Summary of the Invention

[0004] In view of this, the present invention provides a perimeter security method, device, terminal and readable storage medium to solve the problem that current perimeter intrusion prevention solutions have limited application scenarios and cannot be migrated to new scenarios and new target categories for detection in a low-cost and fast manner.

[0005] To solve the above-mentioned technical problems, in a first aspect, the present invention provides a perimeter security method, the method comprising:

[0006] The target segmentation network is trained using a public dataset based on perimeter security to obtain the trained target segmentation network.

[0007] A category database is constructed based on the on-site images of the target site and the trained target segmentation network;

[0008] The trained target segmentation network is used to extract the target feature vector of the target video frame image of the target site.

[0009] Based on the target feature vector, a search is performed in the category database to determine whether there is a feature prototype that matches the target feature vector;

[0010] If the search result shows that a feature prototype matching the target feature vector exists, then based on the pixel information of the target object corresponding to the target feature vector and the warning outline set in the target site, it is determined whether the target object has invaded the target site.

[0011] Optionally, constructing the category database based on the on-site images of the target site and the trained target segmentation network includes:

[0012] The on-site images are classified and dimensionality reduced to obtain dimensionality-reduced feature vectors;

[0013] Based on the reduced-dimensionality feature vector, determine the feature prototype of each target category;

[0014] Each of the aforementioned feature prototypes is iteratively quantized to generate a binary code corresponding to the feature prototype;

[0015] A category database is constructed based on the binary encoding.

[0016] Optionally, the classification and dimensionality reduction processing of the scene images to obtain the dimensionality-reduced feature vector includes:

[0017] Determine the target category to be detected, and classify the scene images according to the target category;

[0018] A binary mask image is created for each of the aforementioned scene images, and the binary mask image is stored as a label in the same classification logic as the target category to which the scene image belongs.

[0019] The scene image is used as the input to the embedding layer of the trained target segmentation network, and the output is a fully connected layer feature vector.

[0020] The feature vector of the fully connected layer is fused with the label and then subjected to masked average pooling to obtain a dimensionality-reduced feature vector.

[0021] Optionally, determining the feature prototype of each target category based on the dimensionality-reduced feature vector includes:

[0022] If there is only one scene image of the same target category, then the dimensionality reduction feature vector corresponding to the scene image is used as the feature prototype of the target category.

[0023] If the scene images of the same target category include at least two images, then the cluster center points of the dimensionality reduction feature vectors corresponding to the scene images of the same target category are used as the feature prototypes of the target category.

[0024] Optionally, the step of searching the category database based on the target feature vector to determine whether a feature prototype matching the target feature vector exists includes:

[0025] The target feature vector is cosine fused with the feature prototypes in the category database to obtain a binary mask.

[0026] Based on the binary mask, obtain a screenshot of the target region of the target video frame image;

[0027] The target region screenshot is iteratively quantized to generate a target code;

[0028] The distance between the target code and the binary code corresponding to the feature prototype in the category database is calculated.

[0029] Based on the distance calculation results, determine whether there exists a feature prototype that matches the target feature vector.

[0030] Optionally, determining whether the target object has intruded into the target site based on the pixel information of the target object corresponding to the target feature vector and the warning outline set within the target site includes:

[0031] If the ratio of the number of pixels inside the warning outline to the total number of pixels in the target object is greater than a preset threshold, then it is determined that the target object in the target video frame image has intruded into the target site.

[0032] Secondly, the present invention also provides a perimeter security device, the device comprising:

[0033] The training module is used to train the target segmentation network on a public dataset based on perimeter security, and obtain the trained target segmentation network.

[0034] The construction module is used to build a category database based on the on-site images of the target site and the trained target segmentation network;

[0035] The extraction module is used to extract the target feature vector of the target video frame image of the target site using the trained target segmentation network;

[0036] The retrieval module is used to search the category database based on the target feature vector to determine whether there is a feature prototype that matches the target feature vector.

[0037] The intrusion determination module is used to determine whether the target object has intruded into the target site if the search result shows that there is a feature prototype that matches the target feature vector, based on the pixel information of the target object corresponding to the target feature vector and the warning outline set in the target site.

[0038] Optionally, the building module includes:

[0039] The dimensionality reduction unit is used to classify and reduce the dimensionality of the scene images to obtain dimensionality-reduced feature vectors;

[0040] The determining unit is used to determine the feature prototype of each target category based on the dimensionality-reduced feature vector;

[0041] An iterative quantization unit is used to iteratively quantize each of the feature prototypes to generate a binary code corresponding to the feature prototype.

[0042] A construction unit is used to construct a category database based on the binary encoding.

[0043] Optionally, the dimensionality reduction unit includes:

[0044] A classification subunit is used to determine the category of the target to be detected and to classify the scene image according to the target category;

[0045] The encoding subunit is used to create a binary mask image for each of the scene images, and store the binary mask image as a label and the target category to which the scene image belongs according to the same classification logic;

[0046] The processing subunit is used to take the scene image as the input to the embedding layer of the trained target segmentation network and output the feature vector of the fully connected layer.

[0047] The dimensionality reduction subunit is used to fuse the feature vector of the fully connected layer with the label and then perform masked average pooling to obtain the dimensionality reduction feature vector.

[0048] Optionally, the determining unit includes:

[0049] The first determining subunit is used to take the dimensionality reduction feature vector corresponding to the scene image as the feature prototype of the target category if there is only one scene image of the same target category.

[0050] The second determining subunit is used to, if the scene images of the same target category include at least two images, use the cluster center point of the dimensionality reduction feature vector corresponding to the scene images of the same target category as the feature prototype of the target category.

[0051] Optionally, the retrieval module includes:

[0052] The cosine fusion unit is used to perform cosine fusion between the target feature vector and the feature prototype in the category database to obtain a binary mask.

[0053] The acquisition unit is used to acquire a screenshot of the target region of the target video frame image based on the binary mask.

[0054] An iterative quantization unit is used to iteratively quantize the target region screenshot to generate a target code;

[0055] The calculation unit is used to calculate the distance between the target code and the binary code corresponding to the feature prototype in the category database;

[0056] The judgment unit is used to determine whether there is a feature prototype that matches the target feature vector based on the distance calculation result.

[0057] Optionally, the intrusion determination module includes:

[0058] An intrusion determination unit is used to determine that the target object in the target video frame image has intruded into the target site if the ratio of the number of pixels inside the warning outline of the target object to the total number of pixels of the target object is greater than a preset threshold.

[0059] Thirdly, the present invention also provides a terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor; when the processor executes the computer program, it implements any of the above-described perimeter security methods.

[0060] Fourthly, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in any of the above-described perimeter protection methods.

[0061] The beneficial effects of the above-described technical solution of the present invention are as follows:

[0062] In this embodiment of the invention, by using a public dataset to train the target segmentation network and constructing a category database, the dependence on the training dataset is reduced. Users can freely construct the category database according to specific scenarios, making the perimeter prevention method of the present invention adaptable to various scenarios, speeding up the retrieval speed, and making the perimeter intrusion judgment more accurate. Attached Figure Description

[0063] Figure 1 This is a flowchart illustrating a perimeter security method provided in Embodiment 1 of the present invention;

[0064] Figure 2 A schematic diagram illustrating the target segmentation and retrieval process provided in Embodiment 1 of this application;

[0065] Figure 3 This is a schematic diagram of the structure of a perimeter security device provided in Embodiment 2 of the present invention;

[0066] Figure 4 This is a schematic diagram of the structure of a terminal according to Embodiment 3 of the present invention. Detailed Implementation

[0067] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the described embodiments of the present invention are within the scope of protection of the present invention.

[0068] Please see Figure 1 , Figure 1 This is a flowchart illustrating a perimeter security method provided in Embodiment 1 of the present invention. The method is applied to a terminal and includes the following steps:

[0069] Step 11: Train the target segmentation network based on the public dataset of perimeter prevention to obtain the trained target segmentation network;

[0070] In this embodiment, the public dataset refers to datasets on the Internet. Since the target segmentation network is applied to the field of perimeter security, the public dataset used is also data from the perimeter security field, such as data from construction site perimeter security. The target segmentation network is a deep learning neural network used to achieve target segmentation, typically including structures such as encoding networks and decoding networks.

[0071] Optionally, the target segmentation network in this embodiment incorporates an iterative quantization loss function to improve its accuracy.

[0072] Step 12: Construct a category database based on the on-site images of the target site and the trained target segmentation network;

[0073] In this embodiment, the target site is the area requiring perimeter security. This target site can be a construction site, etc., and its location and size can be determined according to the perimeter security requirements. The on-site images of the target site are pictures containing the target site's contents. These on-site images can be obtained by taking photos manually at the target site, and they typically cover different angles of the target site.

[0074] The scene images can contain targets to be detected, such as pedestrians and vehicles. Scene images containing pedestrians are all data under the pedestrian category, while scene data containing vehicles are all data under the vehicle category. By using scene images of different categories, a category database can be constructed. The category database includes data of different categories, that is, feature prototypes of different target categories.

[0075] Step 13: Use the trained target segmentation network to extract the target feature vector of the target video frame image of the target site;

[0076] In this embodiment, to automatically implement perimeter protection for the target area, surveillance cameras can be installed within or near the target area, with the cameras' viewing angles covering the entire target area. Subsequently, a trained target segmentation network can be used to perform frame-by-frame detection and analysis on the captured video of the target area to extract the target feature vector from each video frame.

[0077] Step 14: Based on the target feature vector, search the category database to determine whether there exists a feature prototype that matches the target feature vector;

[0078] After extracting the target feature vector from a target video frame image, the target feature vector can be used to search in a category database to determine whether a matching feature prototype exists in the category database.

[0079] Step 15: If the search result shows that there is a feature prototype that matches the target feature vector, then determine whether the target object has invaded the target site based on the pixel information of the target object corresponding to the target feature vector and the warning outline set in the target site.

[0080] In this embodiment, if a feature prototype matching the target feature vector extracted from the target video frame image is found in the category database, it indicates that there is a target object in the target video frame image that requires perimeter protection, and this target object corresponds to the target feature vector. Therefore, based on the pixel information of the target object detected from the target video frame image and the warning outline set within the target area, it can be determined whether the target object has intruded into the target area. The pixel information of the target object can determine its posture, volume, position, etc., and the warning outline is the boundary line of the target area that needs to be guarded and protected. If the target object is located within the warning outline, it is determined that the target object has intruded into the target area; if the target object is located outside the warning outline, it is determined that the target object has not intruded into the target area.

[0081] Therefore, the perimeter defense method provided in this embodiment of the invention reduces the dependence on the training dataset by using a public dataset to train a target segmentation network and construct a category database. Users can freely construct the category database according to specific scenarios, making the perimeter defense method of this invention adaptable to various scenarios, speeding up the retrieval speed, and making the perimeter intrusion judgment more accurate.

[0082] The following examples illustrate the perimeter security methods described above.

[0083] In one optional implementation, the step of constructing a category database based on the on-site images of the target site and the trained target segmentation network includes:

[0084] The on-site images are classified and dimensionality reduced to obtain dimensionality-reduced feature vectors;

[0085] Based on the reduced-dimensionality feature vector, determine the feature prototype of each target category;

[0086] Each of the aforementioned feature prototypes is iteratively quantized to generate a binary code corresponding to the feature prototype;

[0087] A category database is constructed based on the binary encoding.

[0088] In this embodiment, when constructing the category database, firstly, the captured scene images are classified, for example, pedestrians are one category and vehicles are another, and all scene images containing pedestrians belong to the pedestrian category. Next, the scene images are subjected to dimensionality reduction processing to obtain the corresponding dimensionality-reduced feature vectors. Then, the feature prototypes of each target category are determined based on the obtained dimensionality-reduced feature vectors. That is, the corresponding feature prototypes of each target category to be detected need to be determined. In order to reduce errors and improve retrieval efficiency, each feature prototype can be further iteratively quantized so that the data points are mapped to the vertices of a binary hypercube, so that the corresponding quantization error is minimized, thereby obtaining a good binary code corresponding to the feature prototype. Finally, the category database can be constructed based on the binary code, and the category database contains the binary code corresponding to the feature prototype of each target category.

[0089] In this embodiment, the category database is used to retrieve targets in subsequent target video frame images to determine whether there are targets in the target video frame images that require perimeter defense assessment. Since the category database construction in this embodiment employs an iterative quantization algorithm, and the data originates from on-site images, for different defense scenarios, only the corresponding on-site images need to be changed to automatically construct the corresponding category database, achieving rapid scenario migration. Furthermore, the different categories of objects requiring defense can be determined by the user, facilitating the addition of new category objects or the deletion of existing category objects.

[0090] In other embodiments of this application, the classification and dimensionality reduction processing of the scene images to obtain dimensionality-reduced feature vectors includes:

[0091] Determine the target category to be detected, and classify the scene images according to the target category;

[0092] A binary mask image is created for each of the aforementioned scene images, and the binary mask image is stored as a label in the same classification logic as the target category to which the scene image belongs.

[0093] The scene image is used as the input to the embedding layer of the trained target segmentation network, and the output is a fully connected layer feature vector.

[0094] The feature vector of the fully connected layer is fused with the label and then subjected to masked average pooling to obtain a dimensionality-reduced feature vector.

[0095] In this embodiment, the user can define the target category to be detected. The target category can be determined according to the specific scenario. For example, when the target site is a construction site, the target categories to be detected include pedestrians, vehicles, etc. Then, the scene images are classified according to the defined target categories. For each scene image used to build the category database, a binary mask image is created using it, and this binary mask image is stored as a label and the target category of the scene image according to the same classification logic. Then, the embedding layer (VGG network) of the trained target segmentation network is extracted, and the scene image is used as the input of the embedding layer. Finally, the feature vector of its last fully connected layer is output. Finally, the output fully connected layer feature vector is fused with the label and subjected to mask average pooling to obtain the corresponding dimensionality-reduced feature vector.

[0096] In some embodiments of this application, determining the feature prototype of each target category based on the dimensionality-reduced feature vector includes:

[0097] If there is only one scene image of the same target category, then the dimensionality reduction feature vector corresponding to the scene image is used as the feature prototype of the target category.

[0098] If the scene images of the same target category include at least two images, then the cluster center points of the dimensionality reduction feature vectors corresponding to the scene images of the same target category are used as the feature prototypes of the target category.

[0099] In this embodiment, if there is only one scene image under the same target category, the reduced feature vector corresponding to that scene image can be directly used as the feature prototype of that target category; if there are multiple scene images belonging to the same target category, the cluster center point of the reduced feature vectors corresponding to all scene images belonging to the same target category can be used as the feature prototype of that target category. For example, the cluster center point can be a K-means cluster center point for subsequent retrieval.

[0100] In this embodiment, the detection of moving targets in the real-time video of the target site captured by the camera can be achieved by a target segmentation algorithm. To reduce the data dependence on the algorithm and improve the transferability to new scenes and new target categories, this embodiment adopts a target segmentation algorithm based on Few-shots to detect targets frame by frame in the video, and combines this with searching a category database to achieve the purpose of rapid target detection. That is, the target segmentation model in this embodiment adopts the Few-shot target segmentation model PA-Net with inter-class error correction. The inter-class error correction is used to alleviate the difference between the expected features of the support set and the expected features of the query set. The principle of inter-class error correction is described below.

[0101] Let the query set expect The inter-class error ε is as follows:

[0102]

[0103] in, To support the feature vector of the set, Let i be the feature vector of the query set, j be positive integers, S be the number of images in the support set, and Q be the number of images in the query set.

[0104] To improve accuracy, similarity needs to be increased. However, the support set S and the query set Q have different distributions, which affects the similarity calculation cos(P). i X i，q The key is to introduce inter-class error ε to compensate for it. Suppose that the query set samples can be represented by feature prototype P, that is:

[0105] X = P + ∈

[0106]

[0107] Where N is the number of feature prototypes, P i P represents the feature prototype of the i-th image in the query set. i +∈ i，q +ε can represent the support set. Therefore, this expression is equivalent to minimizing the following objective function:

[0108]

[0109] In other words, the above-mentioned objective function is introduced in the embodiments of this application to realize inter-class error correction, thereby reducing inter-class errors.

[0110] The support set can be understood as the on-site images in the aforementioned embodiments, while the query set can be understood as the target video frame images in the aforementioned embodiments.

[0111] In some embodiments of this application, the step of searching the category database based on the target feature vector to determine whether a feature prototype matching the target feature vector exists includes:

[0112] The target feature vector is cosine fused with the feature prototypes in the category database to obtain a binary mask.

[0113] Based on the binary mask, obtain a screenshot of the target region of the target video frame image;

[0114] The target region screenshot is iteratively quantized to generate a target code;

[0115] The distance between the target code and the binary code corresponding to the feature prototype in the category database is calculated.

[0116] Based on the distance calculation results, determine whether there exists a feature prototype that matches the target feature vector.

[0117] Please refer to Figure 2 , Figure 2 This is a schematic diagram of the target segmentation and retrieval process provided in Embodiment 1 of this application. In this embodiment, the scene image is used as the input to the trained target segmentation network, i.e., the scene image is used as the support set. After obtaining the binary mask image, the fully connected layer feature vector output by the fully connected layer and the binary mask image are fused and then average pooled to obtain the feature prototype. The category database can then be established based on the feature prototype. The target video frame image is used as the input to the target segmentation network, i.e., the target video frame image is used as the query set. The target feature vector (i.e., the query set feature vector) of the target video frame image is extracted using the embedding layer of the trained target segmentation network, and error correction is introduced. The above objective function is used to reduce inter-class errors. Next, cosine fusion (cosine calculation) is performed on the target feature vector and the feature prototypes in the category database to obtain a binary mask for the query set. Then, based on this binary mask, a screenshot of the target region in the query set image (i.e., the target video frame image) is obtained. Iterative quantization calculation is then performed to obtain the query set target code. The distance between this query set target code and the binary code of the support set feature prototypes in the category database is then calculated. Finally, based on the distance calculation result, it is determined whether there is a feature prototype matching the target feature vector. For example, if the distance is less than a certain preset threshold, the similarity between the two is considered high, and a target object requiring perimeter protection appears in the target video frame image. If the distance is greater than a certain preset threshold, the similarity between the two is considered low, and no target object requiring perimeter protection exists in the target video frame image. After completing the detection and judgment of the current target video frame image, the processing of the next frame begins.

[0118] like Figure 2As shown, optionally, the Hanmin distance algorithm can be used to calculate the distance, and the distance calculation result can be placed in the candidate class list. Since there is usually more than one feature prototype in the category database, it is necessary to calculate the distance between the query set target code and each feature prototype in the category database one by one, that is, to determine whether the entire category database has been traversed. If not, the traversal continues (i.e., the distance with the next feature prototype is calculated).

[0119] In other embodiments of this application, determining whether the target object has intruded into the target site based on the pixel information of the target object corresponding to the target feature vector and the warning outline set within the target site includes:

[0120] If the ratio of the number of pixels inside the warning outline to the total number of pixels in the target object is greater than a preset threshold, then it is determined that the target object in the target video frame image has intruded into the target site.

[0121] In this embodiment, after detecting a target object, since perimeter security typically involves delineating a warning zone within the view captured by a camera at a fixed angle, defined by a warning outline, a target object within this zone is considered a perimeter intrusion, while a target object outside the warning zone is considered a non-intrusion. However, when a target object is at the boundary of the warning zone, its posture and size vary, and the camera angle differs, making it difficult to determine intrusion. In this case, the pixel information of the target object corresponding to its feature vector and the warning outline set within the target area can be used to determine whether the target object has intruded into the target area. Specifically, if the ratio of the number of pixels within the warning outline to the total number of pixels in the target object is greater than a preset threshold, the target object in the target video frame is determined to have intruded into the target area; otherwise, the target object has not intruded into the target area.

[0122] For example, the following parameters are set:

[0123] The total number of pixels of target object A in the target video frame image is max_pixel(A);

[0124] The number of pixels within the warning outline of target object A is inner_pixel(A);

[0125] The percentage of pixels within the warning outline of target object A to the total number of pixels, K(A);

[0126] Intrusion threshold X;

[0127] Number of intrusion frames alert(A);

[0128] The total number of frames threshold max_alert(A) for determining that the target object A has invaded;

[0129] For a frame of the target video frame image, after obtaining the pixel information of the target object, the perimeter intrusion determination method is as follows:

[0130] Step 1: Calculate max_pixel(A) and inner_pixel(A), and calculate the ratio K(A);

[0131] Step 2: If K(A)≥X, it is determined that the target object has invaded, and this frame is determined as an intrusion frame, and the number of alert(A) is increased by 1. If K(A)<X and alert(A)>0, then alert(A) is cleared;

[0132] Step 3: If alert(A)≥max_alert(A), it is determined that the target object has crossed the boundary, and an alarm is issued for the target object;

[0133] Step 4: Loop steps 1-3 for each target object until the determination of all target objects in this frame is completed;

[0134] Step 5: Enter the next frame and repeat steps 1-4.

[0135] In the embodiment of the present invention, by using the publicly available dataset to train the target segmentation network and constructing a category database, the dependence on the training dataset is reduced. The user can freely construct the category database according to the specific scenario, so that the perimeter prevention method of the present invention can adapt to multiple scenarios, speed up the retrieval speed, and the perimeter intrusion determination is more accurate.

[0136] Please refer to Figure 3 , Figure 3 is a schematic structural diagram of a perimeter prevention device provided in the second embodiment of the present invention. The device 30 includes:

[0137] A training module 31, configured to train the target segmentation network based on the publicly available dataset for perimeter prevention to obtain the trained target segmentation network;

[0138] A construction module 32, configured to construct a category database according to the on-site pictures of the target site and the trained target segmentation network;

[0139] An extraction module 33, configured to extract the target feature vector of the target video frame image of the target site by using the trained target segmentation network;

[0140] A retrieval module 34, configured to retrieve in the category database according to the target feature vector to determine whether there is a feature prototype matching the target feature vector;

[0141] The intrusion determination module 35 is used to determine whether the target object has intruded into the target site if the search result shows that there is a feature prototype that matches the target feature vector, based on the pixel information of the target object corresponding to the target feature vector and the warning outline set in the target site.

[0142] Optionally, the building module includes:

[0143] The dimensionality reduction unit is used to classify and reduce the dimensionality of the scene images to obtain dimensionality-reduced feature vectors;

[0144] The determining unit is used to determine the feature prototype of each target category based on the dimensionality-reduced feature vector;

[0145] An iterative quantization unit is used to iteratively quantize each of the feature prototypes to generate a binary code corresponding to the feature prototype.

[0146] A construction unit is used to construct a category database based on the binary encoding.

[0147] Optionally, the dimensionality reduction unit includes:

[0148] A classification subunit is used to determine the category of the target to be detected and to classify the scene image according to the target category;

[0149] The encoding subunit is used to create a binary mask image for each of the scene images, and store the binary mask image as a label and the target category to which the scene image belongs according to the same classification logic;

[0150] The processing subunit is used to take the scene image as the input to the embedding layer of the trained target segmentation network and output the feature vector of the fully connected layer.

[0151] The dimensionality reduction subunit is used to fuse the feature vector of the fully connected layer with the label and then perform masked average pooling to obtain the dimensionality reduction feature vector.

[0152] Optionally, the determining unit includes:

[0153] The first determining subunit is used to take the dimensionality reduction feature vector corresponding to the scene image as the feature prototype of the target category if there is only one scene image of the same target category.

[0154] The second determining subunit is used to, if the scene images of the same target category include at least two images, use the cluster center point of the dimensionality reduction feature vector corresponding to the scene images of the same target category as the feature prototype of the target category.

[0155] Optionally, the retrieval module includes:

[0156] The cosine fusion unit is used to perform cosine fusion between the target feature vector and the feature prototype in the category database to obtain a binary mask.

[0157] The acquisition unit is used to acquire a screenshot of the target region of the target video frame image based on the binary mask.

[0158] An iterative quantization unit is used to iteratively quantize the target region screenshot to generate a target code;

[0159] The calculation unit is used to calculate the distance between the target code and the binary code corresponding to the feature prototype in the category database;

[0160] The judgment unit is used to determine whether there is a feature prototype that matches the target feature vector based on the distance calculation result.

[0161] Optionally, the intrusion determination module includes:

[0162] An intrusion determination unit is used to determine that the target object in the target video frame image has intruded into the target site if the ratio of the number of pixels inside the warning outline of the target object to the total number of pixels of the target object is greater than a preset threshold.

[0163] In this embodiment of the invention, by using a public dataset to train the target segmentation network and constructing a category database, the dependence on the training dataset is reduced. Users can freely construct the category database according to specific scenarios, making the perimeter prevention method of the present invention adaptable to various scenarios, speeding up the retrieval speed, and making the perimeter intrusion judgment more accurate.

[0164] The embodiments of the present invention are product embodiments corresponding to the method embodiment one described above, and therefore will not be repeated here. For details, please refer to the embodiment one described above.

[0165] Please see Figure 4 , Figure 4 This is a schematic diagram of a terminal structure provided in Embodiment 3 of the present invention. The terminal 40 includes a processor 41, a memory 42, and a computer program stored in the memory 42 and executable on the processor 41. When the processor 41 executes the computer program, it performs the following steps:

[0166] The target segmentation network is trained using a public dataset based on perimeter security to obtain the trained target segmentation network.

[0167] A category database is constructed based on the on-site images of the target site and the trained target segmentation network;

[0168] The trained target segmentation network is used to extract the target feature vector of the target video frame image of the target site.

[0169] Based on the target feature vector, a search is performed in the category database to determine whether there is a feature prototype that matches the target feature vector;

[0170] If the search result shows that a feature prototype matching the target feature vector exists, then based on the pixel information of the target object corresponding to the target feature vector and the warning outline set in the target site, it is determined whether the target object has invaded the target site.

[0171] In this embodiment of the invention, optionally, when the processor 41 executes the computer program, it may also perform the following steps:

[0172] The step of constructing a category database based on on-site images of the target site and the trained target segmentation network includes:

[0173] The on-site images are classified and dimensionality reduced to obtain dimensionality-reduced feature vectors;

[0174] Based on the reduced-dimensionality feature vector, determine the feature prototype of each target category;

[0175] Each of the aforementioned feature prototypes is iteratively quantized to generate a binary code corresponding to the feature prototype;

[0176] A category database is constructed based on the binary encoding.

[0177] Optionally, the classification and dimensionality reduction processing of the scene images to obtain the dimensionality-reduced feature vector includes:

[0178] Determine the target category to be detected, and classify the scene images according to the target category;

[0179] A binary mask image is created for each of the aforementioned scene images, and the binary mask image is stored as a label in the same classification logic as the target category to which the scene image belongs.

[0180] The scene image is used as the input to the embedding layer of the trained target segmentation network, and the output is a fully connected layer feature vector.

[0181] The feature vector of the fully connected layer is fused with the label and then subjected to masked average pooling to obtain a dimensionality-reduced feature vector.

[0182] Optionally, determining the feature prototype of each target category based on the dimensionality-reduced feature vector includes:

[0183] If there is only one scene image of the same target category, then the dimensionality reduction feature vector corresponding to the scene image is used as the feature prototype of the target category.

[0184] If the scene images of the same target category include at least two images, then the cluster center points of the dimensionality reduction feature vectors corresponding to the scene images of the same target category are used as the feature prototypes of the target category.

[0185] Optionally, the step of searching the category database based on the target feature vector to determine whether a feature prototype matching the target feature vector exists includes:

[0186] The target feature vector is cosine fused with the feature prototypes in the category database to obtain a binary mask.

[0187] Based on the binary mask, obtain a screenshot of the target region of the target video frame image;

[0188] The target region screenshot is iteratively quantized to generate a target code;

[0189] The distance between the target code and the binary code corresponding to the feature prototype in the category database is calculated.

[0190] Based on the distance calculation results, determine whether there exists a feature prototype that matches the target feature vector.

[0191] Optionally, determining whether the target object has intruded into the target site based on the pixel information of the target object corresponding to the target feature vector and the warning outline set within the target site includes:

[0192] If the ratio of the number of pixels inside the warning outline to the total number of pixels in the target object is greater than a preset threshold, then it is determined that the target object in the target video frame image has intruded into the target site.

[0193] In this embodiment of the invention, by using a public dataset to train the target segmentation network and constructing a category database, the dependence on the training dataset is reduced. Users can freely construct the category database according to specific scenarios, making the perimeter prevention method of the present invention adaptable to various scenarios, speeding up the retrieval speed, and making the perimeter intrusion judgment more accurate.

[0194] The specific working process of this invention embodiment is the same as that in the method embodiment one above, so it will not be repeated here. For details, please refer to the description of the method steps in the method embodiment one above.

[0195] Embodiment 4 of the present invention provides a computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the steps of any of the perimeter security methods in Embodiment 1 above. For details, please refer to the description of the method steps in the corresponding embodiments above.

[0196] The aforementioned computer-readable storage media include both permanent and non-permanent, removable and non-removable media, which can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0197] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A perimeter security method, characterized in that, include: The target segmentation network is trained using a public dataset based on perimeter security to obtain the trained target segmentation network. A category database is constructed based on the on-site images of the target site and the trained target segmentation network; The trained target segmentation network is used to extract the target feature vector of the target video frame image of the target site. Based on the target feature vector, a search is performed in the category database to determine whether there is a feature prototype that matches the target feature vector; If the search result shows that there is a feature prototype that matches the target feature vector, then based on the pixel information of the target object corresponding to the target feature vector and the warning outline set in the target site, it is determined whether the target object has invaded the target site. The step of searching the category database based on the target feature vector to determine whether a feature prototype matching the target feature vector exists includes: The target feature vector is cosine fused with the feature prototypes in the category database to obtain a binary mask. Based on the binary mask, obtain a screenshot of the target region of the target video frame image; The target region screenshot is iteratively quantized to generate a target code; The distance between the target code and the binary code corresponding to the feature prototype in the category database is calculated. Based on the distance calculation results, determine whether there exists a feature prototype that matches the target feature vector.

2. The method according to claim 1, characterized in that, The step of constructing a category database based on on-site images of the target site and the trained target segmentation network includes: The on-site images are classified and dimensionality reduced to obtain dimensionality-reduced feature vectors; Based on the reduced-dimensionality feature vector, determine the feature prototype of each target category; Each of the aforementioned feature prototypes is iteratively quantized to generate a binary code corresponding to the feature prototype; A category database is constructed based on the binary encoding.

3. The method according to claim 2, characterized in that, The classification and dimensionality reduction processing of the scene images to obtain the dimensionality-reduced feature vector includes: Determine the target category to be detected, and classify the scene images according to the target category; A binary mask image is created for each of the aforementioned scene images, and the binary mask image is stored as a label in the same classification logic as the target category to which the scene image belongs. The scene image is used as the input to the embedding layer of the trained target segmentation network, and the output is a fully connected layer feature vector. The feature vector of the fully connected layer is fused with the label and then subjected to masked average pooling to obtain a dimensionality-reduced feature vector.

4. The method according to any one of claims 2 or 3, characterized in that, The step of determining the feature prototype of each target category based on the dimensionality-reduced feature vector includes: If there is only one scene image of the same target category, then the dimensionality reduction feature vector corresponding to the scene image is used as the feature prototype of the target category. If the scene images of the same target category include at least two images, then the cluster center points of the dimensionality reduction feature vectors corresponding to the scene images of the same target category are used as the feature prototypes of the target category.

5. The method according to claim 1, characterized in that, The step of determining whether the target object has intruded into the target site based on the pixel information of the target object corresponding to the target feature vector and the warning outline set within the target site includes: If the ratio of the number of pixels inside the warning outline to the total number of pixels in the target object is greater than a preset threshold, then it is determined that the target object in the target video frame image has intruded into the target site.

6. A perimeter security device, characterized in that, include: The training module is used to train the target segmentation network on a public dataset based on perimeter security, and obtain the trained target segmentation network. The construction module is used to build a category database based on the on-site images of the target site and the trained target segmentation network; The extraction module is used to extract the target feature vector of the target video frame image of the target site using the trained target segmentation network; The retrieval module is used to search the category database based on the target feature vector to determine whether there is a feature prototype that matches the target feature vector. The intrusion determination module is used to determine whether the target object has intruded into the target site if the search result shows that there is a feature prototype that matches the target feature vector, based on the pixel information of the target object corresponding to the target feature vector and the warning outline set in the target site. The step of searching the category database based on the target feature vector to determine whether a feature prototype matching the target feature vector exists includes: The target feature vector is cosine fused with the feature prototypes in the category database to obtain a binary mask. Based on the binary mask, obtain a screenshot of the target region of the target video frame image; The target region screenshot is iteratively quantized to generate a target code; The distance between the target code and the binary code corresponding to the feature prototype in the category database is calculated. Based on the distance calculation results, determine whether there exists a feature prototype that matches the target feature vector.

7. The apparatus according to claim 6, characterized in that, The building module includes: The dimensionality reduction unit is used to classify and reduce the dimensionality of the scene images to obtain dimensionality-reduced feature vectors; The determining unit is used to determine the feature prototype of each target category based on the reduced feature vector; An iterative quantization unit is used to iteratively quantize each of the feature prototypes to generate a binary code corresponding to the feature prototype. A construction unit is used to construct a category database based on the binary encoding.

8. A terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor; characterized in that, When the processor executes the computer program, it implements the perimeter security method as described in any one of claims 1 to 5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the perimeter security method as described in any one of claims 1 to 5.

Images