ATM cabinet inspection method and device
By combining feature extraction and fusion of real and virtual images, the problem of insufficient data in ATM cabinet inspection is solved, and the detection accuracy and accuracy are improved.
Patent Information
- Application Number
- CN202411724962.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-28
- Publication Date
- 2025-02-18
AI Technical Summary
The existing ATM cabinet inspection methods rely on real data to train intelligent models, but it is difficult to collect a large amount of data in damaged scenarios, resulting in inaccurate detection results.
By obtaining the real and virtual images of the ATM cabinet, using a deep learning model for feature extraction, combining the regional candidate network to generate candidate areas, and performing feature fusion, to achieve accurate inspection of the ATM cabinet.
It improves the detection accuracy and inspection accuracy of ATM cabinet damage detection, especially when the number of real data is small.
Smart Images
Figure CN119478547A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of ATM cabinet inspection, and more specifically, to an ATM cabinet inspection method and device. Background Art
[0002] Automated Teller Machine (ATM) is an important offline machine of the bank. ATM uses magnetic code or smart card to realize self-service financial transaction, replacing the work of bank counter staff. In the process of using ATM cabinet, it is inevitable that it will be damaged, such as rust, stains, cracks, etc., which will produce a bad experience for users. Therefore, it is necessary to inspect the ATM cabinet.
[0003] The existing inspection of ATM cabinets mostly relies on real data to train intelligent models, and then inspect the ATM cabinets through the trained intelligent models. However, in the scenario of damaged ATM cabinets, it is difficult to collect a large number of data samples and the data is not comprehensive. In addition, since the real images are manually labeled, there are situations such as fatigue and laziness, which easily lead to wrong labeling and missing labeling of data, resulting in inaccurate results of ATM cabinet inspections, thereby reducing the accuracy of ATM cabinet damage inspections.
[0004] Therefore, how to improve the accuracy of ATM cabinet damage inspection is a problem that needs to be solved urgently in this application. Summary of the invention
[0005] In view of this, the present application discloses an ATM cabinet inspection method and device, aiming to improve the detection accuracy of ATM cabinet damage detection, thereby improving the accuracy of ATM cabinet damage inspection.
[0006] In order to achieve the above purpose, the disclosed technical solution is as follows:
[0007] The first aspect of the present application discloses an ATM cabinet inspection method, the method comprising:
[0008] Obtaining a real image of the ATM cabinet to be detected and a virtual image corresponding to the ATM cabinet to be detected; the virtual image is used to represent the virtual style of the ATM counter to be detected;
[0009] Performing feature extraction on the real image and the virtual image of the ATM cabinet to be inspected through a deep learning model to obtain real features and virtual features;
[0010] Generate candidate regions of the ATM cabinet to be detected through the region candidate network;
[0011] The candidate area, the real feature and the virtual feature are subjected to feature fusion to obtain a feature representation method of the ATM cabinet to be detected;
[0012] Through the deep learning model, the feature representation method of the ATM cabinet to be inspected and the candidate area are classified and position regressed to complete the inspection of the ATM cabinet.
[0013] Preferably, the obtaining of the real image of the ATM cabinet to be detected and the virtual image corresponding to the ATM cabinet to be detected includes:
[0014] Acquire the real image of the ATM cabinet to be inspected through the preset acquisition device;
[0015] Create a virtual environment;
[0016] In the virtual environment, a rendering operation is performed on the ATM cabinet to be detected by a rendering engine to obtain a virtual image corresponding to the ATM cabinet to be detected.
[0017] Preferably, generating a candidate area of the ATM cabinet to be detected through the area candidate network includes:
[0018] The feature extraction of the real image of the ATM cabinet to be detected is performed through the region candidate network to obtain the candidate region feature map;
[0019] At each position on the candidate region feature map, a set of predefined anchor boxes is generated;
[0020] For each anchor box, classification and regression operations are performed through the region proposal network to obtain the initial candidate region;
[0021] Through non-maximum suppression, redundant candidate regions are removed from the initial candidate regions to obtain candidate regions.
[0022] Preferably, the feature fusion of the candidate area, the real feature and the virtual feature to obtain a feature representation method of the ATM cabinet to be detected includes:
[0023] Constructing a category prototype corresponding to the real feature, and constructing a category prototype corresponding to the virtual feature;
[0024] Establishing a consistency constraint between the category prototype corresponding to the real feature and the category prototype corresponding to the virtual feature;
[0025] Under the condition of the consistency constraint, a feature vector corresponding to a candidate box is generated through a regional candidate network, and the feature vector corresponding to the candidate box, the real feature and the virtual feature are feature fused to obtain a feature representation method of the ATM cabinet to be detected.
[0026] Preferably, the deep learning model is used to classify and regress the characteristic representation mode of the ATM cabinet to be inspected and the candidate area to complete the inspection of the ATM cabinet, including:
[0027] The feature representation method of the ATM cabinet to be detected and the candidate area are integrated through a fully connected layer in a deep learning model to obtain an integrated feature vector;
[0028] Multiplying the integrated feature vector and the weight matrix through a fully connected layer in a deep learning model to obtain a product result;
[0029] The product result is output through an activation function through the fully connected layer in the deep learning model to achieve classification and position regression, thereby completing the inspection of the ATM cabinet.
[0030] The second aspect of the present application discloses an ATM cabinet inspection device, the device comprising:
[0031] An acquisition unit is used to acquire a real image of the ATM cabinet to be detected and a virtual image corresponding to the ATM cabinet to be detected; the virtual image is used to represent the virtual style of the ATM counter to be detected;
[0032] A feature extraction unit, used for extracting features from the real image and the virtual image of the ATM cabinet to be detected through a deep learning model to obtain real features and virtual features;
[0033] A generating unit, used for generating a candidate region of an ATM cabinet to be detected through a region candidate network;
[0034] A feature fusion unit, used for fusing the candidate area, the real feature and the virtual feature to obtain a feature representation method of the ATM cabinet to be detected;
[0035] The inspection unit is used to classify and regress the feature representation method of the ATM cabinet to be inspected and the candidate area through a deep learning model to complete the inspection of the ATM cabinet.
[0036] Preferably, the acquisition unit includes:
[0037] An acquisition module is used to acquire a real image of the ATM cabinet to be inspected through a preset acquisition device;
[0038] Create a module for creating a virtual environment;
[0039] The rendering module is used to perform rendering operation on the ATM cabinet to be detected through a rendering engine in the virtual environment to obtain a virtual image corresponding to the ATM cabinet to be detected.
[0040] Preferably, the generating unit comprises:
[0041] A feature extraction module is used to extract features from the real image of the ATM cabinet to be detected through a region candidate network to obtain a candidate region feature map;
[0042] A generation module, used for generating a set of predefined anchor boxes at each position on the candidate region feature map;
[0043] The operation module is used to perform classification and regression operations on each anchor box through the region candidate network to obtain the initial candidate region;
[0044] The removal module is used to remove redundant candidate regions from the initial candidate regions through non-maximum suppression to obtain candidate regions.
[0045] Preferably, the feature fusion unit comprises:
[0046] A construction module, used to construct a category prototype corresponding to a real feature and a category prototype corresponding to a virtual feature;
[0047] An establishing module, used for establishing consistency constraints between the category prototype corresponding to the real feature and the category prototype corresponding to the virtual feature;
[0048] The feature fusion module is used to generate a feature vector corresponding to a candidate box through a regional candidate network under the condition of the consistency constraint, and to fuse the feature vector corresponding to the candidate box, the real feature and the virtual feature to obtain a feature representation method of the ATM cabinet to be detected.
[0049] Preferably, the inspection unit comprises:
[0050] An integration module, used to integrate the feature representation method of the ATM cabinet to be detected and the candidate area through a fully connected layer in a deep learning model to obtain an integrated feature vector;
[0051] A product module, used for multiplying the integrated feature vector and the weight matrix through a fully connected layer in a deep learning model to obtain a product result;
[0052] The inspection module is used to output the product result through an activation function through a fully connected layer in a deep learning model to achieve classification and position regression, thereby completing the inspection of the ATM cabinet.
[0053] Through the above technology, it can be known that the present application discloses an ATM cabinet inspection method and device, which obtains a real image of the ATM cabinet to be inspected and a virtual image corresponding to the ATM cabinet to be inspected, wherein the virtual image is used to represent the virtual style of the ATM counter to be inspected, and a deep learning model is used to extract features from the real image and the virtual image of the ATM cabinet to be inspected to obtain real features and virtual features, and a candidate region of the ATM cabinet to be inspected is generated through a region candidate network, and the candidate region, the real feature and the virtual feature are feature-fused to obtain a feature representation method of the ATM cabinet to be inspected. Through the deep learning model, the feature representation method of the ATM cabinet to be inspected and the candidate region are classified and positionally regressed to complete the inspection of the ATM cabinet.
[0054] Through the above scheme, in the scenario of ATM cabinet damage detection, the amount of real data is relatively small, and the effect of directly using the data to train the target detection model is poor. In order to more effectively use virtual data to improve the detection effect of the model in the real scene, a large number of virtual images with high-precision annotations are combined to extract features of the virtual images, and improvements are made on the loss function to fuse the candidate areas and real features with virtual features to obtain the feature representation of the ATM cabinet to be detected, and the consistency of the feature representation of the ATM cabinet to be detected is supervised, so that the model is more in line with the real data while being fully trained with virtual data, thereby realizing the effective training of the deep learning model with a small amount of real data, and the feature representation of the ATM cabinet to be detected and the candidate areas are classified and regressed through the deep learning model to complete the inspection of the ATM cabinet, improve the detection accuracy of the ATM cabinet damage detection, and thus improve the accuracy of the ATM cabinet damage inspection. BRIEF DESCRIPTION OF THE DRAWINGS
[0055] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.
[0056] Figure 1 A schematic diagram of a flow chart of an ATM cabinet inspection method disclosed in an embodiment of the present application;
[0057] Figure 2 A schematic diagram of the structure of the deep learning model disclosed in the embodiment of the present application;
[0058] Figure 3 A schematic diagram of the structure of an ATM cabinet inspection device disclosed in an embodiment of the present application;
[0059] Figure 4 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of the present application. DETAILED DESCRIPTION
[0060] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0061] In this application, the terms "comprises", "comprising" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element.
[0062] As can be seen from the background technology, the existing inspection of ATM cabinets mostly relies on real data to train intelligent models, and the ATM cabinets are inspected through the trained intelligent models. However, in the scenario of ATM cabinet damage, it is difficult to collect a large number of data samples and the data comprehensiveness is poor. In addition, since the real images are manually labeled, there are situations such as fatigue and laziness, which are prone to mislabeling and missing data labeling, resulting in inaccurate results of ATM cabinet inspections, thereby reducing the accuracy of ATM cabinet damage inspections. Therefore, how to improve the accuracy of ATM cabinet damage inspections is a problem that this application urgently needs to solve.
[0063] In order to solve the above problems, the present application discloses an ATM cabinet inspection method and device. In the scenario of ATM cabinet damage detection, the amount of real data is relatively small, and the effect of directly using the data to train the target detection model is poor. In order to more effectively use virtual data to improve the detection effect of the model in the real scene, a large number of virtual images with high-precision annotations are combined to extract features of the virtual images, and improvements are made on the loss function to fuse the candidate areas and real features and virtual features to obtain the feature representation of the ATM cabinet to be detected, and the consistency of the feature representation of the ATM cabinet to be detected is supervised, so that the model is more in line with the real data while being fully trained with virtual data, thereby realizing the effective training of the deep learning model in the case of a small amount of real data, and the feature representation of the ATM cabinet to be detected and the candidate areas are classified and regressed by the deep learning model to complete the inspection of the ATM cabinet, improve the detection accuracy of the ATM cabinet damage detection, and thus improve the accuracy of the ATM cabinet damage detection. The specific implementation method is specifically described by the following embodiments.
[0064] refer to Figure 1 As shown, an ATM cabinet inspection method disclosed in an embodiment of the present application is shown, and the ATM cabinet inspection method mainly includes the following steps:
[0065] S101: Acquire a real image of the ATM cabinet to be detected and a virtual image corresponding to the ATM cabinet to be detected; the virtual image is used to represent the virtual style of the ATM counter to be detected.
[0066] In S101, a real image of the ATM cabinet to be inspected is obtained through a preset acquisition device. A virtual environment is created, and in the virtual environment, a rendering operation is performed on the ATM cabinet to be inspected through a rendering engine to obtain a virtual image corresponding to the ATM cabinet to be inspected.
[0067] Virtual environments can be created through Unreal Engine. Unreal Engine is a real-time simulation engine that can be used to build virtual simulation environments.
[0068] S102: Extract features from the real image and virtual image of the ATM cabinet to be inspected through a deep learning model to obtain real features and virtual features.
[0069] The real image of the ATM cabinet to be inspected and the virtual image corresponding to the ATM cabinet to be inspected are input into the deep learning model (Faster R-CNN) of the high-precision target detection method. The convolutional neural network (CNN) of the deep learning model is used to extract features from the real image and virtual image of the ATM cabinet to be inspected to obtain real features and virtual features.
[0070] Among them, the target detection algorithm can frame the target in a picture and give the target type and the probability of being judged as this type.
[0071] Faster RCNN is a target detection algorithm, and its detection process is as follows:
[0072] 1. Input image: Input the image of the ATM cabinet to be detected into Faster RCNN;
[0073] 2. Feature extraction: Use convolutional neural network (CNN) to extract features from the input image;
[0074] 3. Generate candidate regions: Generate candidate regions using the Region Proposal Network (RPN);
[0075] 4. ROIPooling: Convert the candidate region into a feature map of fixed size;
[0076] 5. Classification and regression: Use the fully connected layer to classify and regress the candidate areas to complete the inspection of the ATM cabinet.
[0077] Specifically, the detection process of FasterRCNN consists of two stages: candidate region generation stage and target detection stage.
[0078] In the candidate region generation stage, FasterRCNN uses the Region Proposal Network (RPN) to generate a series of candidate regions.
[0079] In the target detection stage, Faster RCNN uses ROIPooling to convert the candidate region into a fixed-size feature map, and uses a fully connected layer to classify and regress the candidate region. The outputs of these steps are combined into the final detection result.
[0080] S103: Generate candidate regions of the ATM cabinet to be detected through the RPN network.
[0081] The specific process of generating candidate regions of the ATM cabinet to be detected through the region candidate network is shown in A1-A4.
[0082] A1: The feature extraction of the real image of the ATM cabinet to be inspected is performed through the RPN network to obtain the candidate region feature map.
[0083] A2: Generate a set of predefined anchor boxes at each position on the candidate region feature map.
[0084] A3: For each anchor box, classification and regression operations are performed through the RPN network to obtain the initial candidate region.
[0085] Among them, the classification operation is used to determine whether the anchor box contains the target.
[0086] The regression operation is used to fine-tune the position of the anchor box so that it surrounds the target more accurately.
[0087] The initial candidate region is a candidate region without removing redundant candidate regions.
[0088] A4: Through non-maximum suppression (NMS), redundant candidate regions are removed from the initial candidate regions to obtain candidate regions.
[0089] NMS is used to remove redundant candidate regions from the initial candidate regions to obtain more accurate candidate regions.
[0090] S104: Fusing the candidate area, the real features and the virtual features to obtain a feature representation method of the ATM cabinet to be detected.
[0091] Specifically, the candidate region, real features and virtual features are fused to obtain the feature representation of the ATM cabinet to be detected, as shown in B1-B3.
[0092] B1: Construct the category prototype corresponding to the real feature, and construct the category prototype corresponding to the virtual feature.
[0093] B2: Establish consistency constraints between the category prototypes corresponding to the real features and the category prototypes corresponding to the virtual features.
[0094] B3: Under the condition of consistency constraints, the feature vector corresponding to the candidate box is generated through the RPN network, and the feature vector corresponding to the candidate box, the real feature and the virtual feature are fused to obtain the feature representation method of the ATM cabinet to be detected.
[0095] (1) Feature Fusion:
[0096] In order to better integrate virtual data and real data, this scheme considers constructing category prototypes for virtual features and real features respectively in a single batch, and then establishing consistency constraints between category prototypes.
[0097] In the training process of Faster RCNN, the RPN network is responsible for generating candidate foreground and background. These candidate boxes provide rich instance information and scene information. However, a single candidate box does not include complete category information. In order to achieve more accurate category feature representation, it is necessary to fuse the features of all candidate boxes belonging to a certain category. The structural diagram of the deep learning model of the improved target detection method is shown in the figure below. Figure 2 shown.
[0098] Figure 2 In the example, ROIPooling is used to convert the candidate region into a fixed-size feature map; ResNet50 is a feature extraction model; F real is the real data feature; F vir is a virtual data feature; Class is the target category; Position is the target position; L consistency is the consistency loss function.
[0099] Specifically, each candidate box generated by the RPN network will be processed into a feature vector at a high level. By fusing the feature, the feature representation of the target category in the data set (i.e., the ATM cabinet to be detected) can be obtained. The fusion method of the vector is shown in Formula 1.
[0100]
[0101] Among them, C k is the fusion feature representation method for the kth category in a single batch; N p is the number of candidate boxes; F i is the feature vector representation of the i-th candidate box for the k-th category in a single batch; P ik is the classification probability vector; T is the transpose.
[0102] It should be noted that feature fusion is only used as a regularization term for model learning and is used to construct consistency supervision during training. During testing, the original features are still used for classification and position regression of the RCNN network, and the amount of calculation during model inference is not increased.
[0103] (2)Consistency supervision:
[0104] FasterRCNN is a two-stage target detection method. The network structure consists of three parts: backbone network, RPN and R-CNN.
[0105] The RPN part generates candidate boxes that are independent of the category, and then R-CNN uses the feature vector obtained from the ROI pooling layer to predict the fine position and category. Each stage has a classification error and position regression error. The overall formula is shown in Formula 2.
[0106]
[0107] Among them, L det For the final loss; It is the RPN classification loss, which is used to determine whether the anchor is groundtruth; It is the RPN position regression loss, which is used to fine-tune the position of the anchor; It is the ROI classification loss, which is used to determine the category to which the ROI belongs; It is the ROI position regression loss, which is used to fine-tune the ROI position.
[0108] Based on this error, the previously constructed category prototype is used to introduce a consistency loss function to reduce the difference between real features and virtual features of the same category. Specifically, in each iterative training, the real data and virtual data are sampled 1:1, and the category prototype is extracted by feature fusion, and the Euclidean distance is used as the measure of the difference, as shown in Formula 3.
[0109]
[0110] Among them, L consistency is the consistency loss function; N C for all categories in all datasets; is the fusion feature of the i-th category of the real data; is the fusion feature of the i-th category of virtual data.
[0111] The loss function is used to define the error between a single training example and the true value.
[0112] The overall loss function of the model is shown in Formula 4.
[0113] L all =L det +λL consistency (4)
[0114] Among them, L all is the total loss function of the model; L det For the final loss; L consistency is the consistency loss function; λ is a hyperparameter.
[0115] S105: Classify and regress the feature representation methods and candidate areas of the ATM cabinet to be inspected through a deep learning model to complete the inspection of the ATM cabinet.
[0116] In S105, the feature representation mode and candidate areas of the ATM cabinet to be inspected are classified and position regressed through the fully connected layer in the deep learning model to complete the inspection of the ATM cabinet.
[0117] This method supervises the features of the two data sets at the category feature level, keeping its feature abstraction ability consistent on the two data sets. This allows the model to learn the classification and position at the same time. In the process of learning category classification and position regression using a large amount of virtual data, the model also learns the feature distribution of real data. Compared with other methods, it makes more effective use of virtual data and can achieve better results on the real data test set.
[0118] Specifically, through the deep learning model, the feature representation method and candidate area of the ATM cabinet to be inspected are classified and the position regression is performed to complete the inspection process of the ATM cabinet, as shown in C1-C3.
[0119] C1: The feature representation method of the ATM cabinet to be detected and the candidate area are integrated through the fully connected layer in the deep learning model to obtain the integrated feature vector.
[0120] C2: Multiply the integrated feature vector and the weight matrix through the fully connected layer in the deep learning model to obtain the product result.
[0121] C3: The product result is output through the activation function through the fully connected layer in the deep learning model to achieve classification and position regression, thereby completing the inspection of the ATM cabinet to be inspected.
[0122] In the case of ATM cabinet damage, there is less real data, and the detection effect is poor when only real data or only virtual data is used. Therefore, we combine a large amount of virtual data with high-precision annotations, make improvements on the loss function, supervise the consistency of high-order features of real and virtual data, train intelligent models, and improve the detection accuracy of ATM cabinet damage detection.
[0123] High-order features refer to the features output by the deep model in the later network structure. These features have been refined many times and can express the characteristics of the sample more comprehensively.
[0124] This solution can detect a variety of damage types, such as rust, stains, cracks, etc. This method fuses the high-level semantic features of the model and supervises the consistency of the fused features of virtual and real data, so that the model can be more closely matched to real data while being fully trained with virtual data. Compared with the original method that only uses real data, the detection effect is significantly improved.
[0125] In the scenario of ATM cabinet damage detection, the amount of real data is relatively small, and the effect of directly using this data to train the target detection model is poor. Therefore, after constructing a virtual ATM cabinet damage data set, in order to be able to more effectively use virtual data to improve the detection effect of the model in real scenarios. Specifically, during the model training process, each instance detected by the model represents the model's representation of the target category. Therefore, by fusion of the high-level semantic features of the candidate box corresponding to each instance, the model's representation of this target category is obtained. Then, the consistency supervision of the high-level semantic features is performed on the real and virtual data, and the feature distribution of the real and virtual data in the model is brought closer, so that it can better understand the real data in the process of training with a large amount of virtual data, thereby realizing the effective training of the detection model with a small amount of real data. The following is an introduction from the perspectives of feature fusion and consistency supervision. It should be noted that this scheme is modified on Faster R-CNN.
[0126] Beneficial effects of the embodiments of the present application: In the scenario of ATM cabinet damage detection, the amount of real data is relatively small, and the effect of directly using the data to train the target detection model is poor. In order to more effectively use virtual data to improve the detection effect of the model in the real scene, a large number of virtual images with high-precision annotations are combined to extract features of the virtual images, and improvements are made on the loss function to fuse the candidate areas with real features and virtual features to obtain the feature representation of the ATM cabinet to be detected, and the consistency of the feature representation of the ATM cabinet to be detected is supervised, so that the model is more in line with the real data while being fully trained with virtual data, thereby realizing effective training of the deep learning model with a small amount of real data, and classifying and regressing the feature representation of the ATM cabinet to be detected and the candidate areas through the deep learning model to complete the inspection of the ATM cabinet, improve the detection accuracy of the ATM cabinet damage detection, and thus improve the accuracy of the ATM cabinet damage inspection.
[0127] Based on the above embodiments Figure 1 The disclosed ATM cabinet inspection method, the present application embodiment also discloses an ATM cabinet inspection device, such as Figure 3 As shown, the ATM cabinet inspection device comprises:
[0128] The acquisition unit 301 is used to acquire a real image of the ATM cabinet to be detected and a virtual image corresponding to the ATM cabinet to be detected; the virtual image is used to represent the virtual style of the ATM counter to be detected;
[0129] A feature extraction unit 302 is used to extract features from the real image and the virtual image of the ATM cabinet to be inspected through a deep learning model to obtain real features and virtual features;
[0130] A generating unit 303, used for generating a candidate area of the ATM cabinet to be detected through an area candidate network;
[0131] A feature fusion unit 304 is used to fuse the candidate area, the real feature and the virtual feature to obtain a feature representation method of the ATM cabinet to be detected;
[0132] The inspection unit 305 is used to classify and regress the feature representation methods and candidate areas of the ATM cabinet to be inspected through a deep learning model to complete the inspection of the ATM cabinet.
[0133] Furthermore, the acquisition unit 301 includes:
[0134] An acquisition module is used to acquire a real image of the ATM cabinet to be inspected through a preset acquisition device;
[0135] Create a module for creating a virtual environment;
[0136] The rendering module is used to render the ATM cabinet to be detected through a rendering engine in a virtual environment to obtain a virtual image corresponding to the ATM cabinet to be detected.
[0137] Furthermore, the generating unit 303 includes:
[0138] A feature extraction module is used to extract features from the real image of the ATM cabinet to be detected through a region candidate network to obtain a candidate region feature map;
[0139] A generation module is used to generate a set of predefined anchor boxes at each position on the candidate region feature map;
[0140] The operation module is used to perform classification and regression operations on each anchor box through the region candidate network to obtain the initial candidate region;
[0141] The removal module is used to remove redundant candidate regions from the initial candidate regions through non-maximum suppression to obtain candidate regions.
[0142] Further, including:
[0143] A construction module, used to construct a category prototype corresponding to a real feature and a category prototype corresponding to a virtual feature;
[0144] Establishing a module for establishing consistency constraints between the category prototypes corresponding to the real features and the category prototypes corresponding to the virtual features;
[0145] The feature fusion module is used to generate a feature vector corresponding to a candidate box through a regional candidate network under the condition of consistency constraints, and to fuse the feature vector corresponding to the candidate box, the real feature and the virtual feature to obtain a feature representation method for the ATM cabinet to be detected.
[0146] Furthermore, the inspection unit 305 includes:
[0147] An integration module, used to integrate the feature representation method of the ATM cabinet to be detected and the candidate area through a fully connected layer in a deep learning model to obtain an integrated feature vector;
[0148] The product module is used to multiply the integrated feature vector and the weight matrix through the fully connected layer in the deep learning model to obtain the product result;
[0149] The inspection module is used to output the product result through an activation function through a fully connected layer in a deep learning model to achieve classification and position regression, thereby completing the inspection of the ATM cabinet.
[0150] Beneficial effects of the embodiments of the present application: In the scenario of ATM cabinet damage detection, the amount of real data is relatively small, and the effect of directly using the data to train the target detection model is poor. In order to more effectively use virtual data to improve the detection effect of the model in the real scene, a large number of virtual images with high-precision annotations are combined to extract features of the virtual images, and improvements are made on the loss function to fuse the candidate areas with real features and virtual features to obtain the feature representation of the ATM cabinet to be detected, and the consistency of the feature representation of the ATM cabinet to be detected is supervised, so that the model is more in line with the real data while being fully trained with virtual data, thereby realizing effective training of the deep learning model with a small amount of real data, and classifying and regressing the feature representation of the ATM cabinet to be detected and the candidate areas through the deep learning model to complete the inspection of the ATM cabinet, improve the detection accuracy of the ATM cabinet damage detection, and thus improve the accuracy of the ATM cabinet damage inspection.
[0151] An embodiment of the present application further provides a storage medium, which includes stored instructions, wherein when the instructions are executed, the device where the storage medium is located is controlled to execute the above-mentioned ATM cabinet inspection method.
[0152] The present application also provides an electronic device, whose structural diagram is as follows: Figure 4As shown, it specifically includes a memory 401 and one or more instructions 402, wherein the one or more instructions 402 are stored in the memory 401 and are configured to be executed by one or more processors 403 to execute the one or more instructions 402 to perform the above-mentioned ATM cabinet inspection method.
[0153] For the aforementioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.
[0154] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.
[0155] The steps in the methods of the embodiments of the present application can be adjusted in order, combined, and deleted according to actual needs.
[0156] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations.
[0157] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.
[0158] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.
Claims
1. A method for inspecting an ATM cabinet, characterized in that: The method comprises: Obtaining a real image of the ATM cabinet to be detected and a virtual image corresponding to the ATM cabinet to be detected; the virtual image is used to represent the virtual style of the ATM counter to be detected; Performing feature extraction on the real image and the virtual image of the ATM cabinet to be inspected through a deep learning model to obtain real features and virtual features; Generate candidate regions of the ATM cabinet to be detected through the region candidate network; The candidate area, the real feature and the virtual feature are subjected to feature fusion to obtain a feature representation method of the ATM cabinet to be detected; Through the deep learning model, the feature representation method of the ATM cabinet to be inspected and the candidate area are classified and position regressed to complete the inspection of the ATM cabinet.
2. The method according to claim 1, characterized in that The obtaining of the real image of the ATM cabinet to be detected and the virtual image corresponding to the ATM cabinet to be detected includes: Acquire the real image of the ATM cabinet to be inspected through the preset acquisition device; Create a virtual environment; In the virtual environment, a rendering operation is performed on the ATM cabinet to be detected by a rendering engine to obtain a virtual image corresponding to the ATM cabinet to be detected.
3. The method according to claim 1, characterized in that The generating of candidate regions of the ATM cabinet to be detected by the region candidate network includes: The feature extraction of the real image of the ATM cabinet to be detected is performed through the region candidate network to obtain the candidate region feature map; At each position on the candidate region feature map, a set of predefined anchor boxes is generated; For each anchor box, classification and regression operations are performed through the region proposal network to obtain the initial candidate region; Through non-maximum suppression, redundant candidate regions are removed from the initial candidate regions to obtain candidate regions.
4. The method according to claim 1, characterized in that: The method of fusing the candidate area, the real feature and the virtual feature to obtain a feature representation method of the ATM cabinet to be detected includes: Constructing a category prototype corresponding to the real feature, and constructing a category prototype corresponding to the virtual feature; Establishing a consistency constraint between the category prototype corresponding to the real feature and the category prototype corresponding to the virtual feature; Under the condition of the consistency constraint, a feature vector corresponding to a candidate box is generated through a regional candidate network, and the feature vector corresponding to the candidate box, the real feature and the virtual feature are feature fused to obtain a feature representation method of the ATM cabinet to be detected.
5. The method according to claim 1, characterized in that The deep learning model is used to classify and regress the feature representation of the ATM cabinet to be inspected and the candidate area to complete the inspection of the ATM cabinet, including: The feature representation method of the ATM cabinet to be detected and the candidate area are integrated through a fully connected layer in a deep learning model to obtain an integrated feature vector; Multiplying the integrated feature vector and the weight matrix through a fully connected layer in a deep learning model to obtain a product result; The product result is output through an activation function through the fully connected layer in the deep learning model to achieve classification and position regression, thereby completing the inspection of the ATM cabinet.
6. An ATM cabinet inspection device, characterized in that: The device comprises: An acquisition unit is used to acquire a real image of the ATM cabinet to be detected and a virtual image corresponding to the ATM cabinet to be detected; the virtual image is used to represent the virtual style of the ATM counter to be detected; A feature extraction unit, used for extracting features from the real image and the virtual image of the ATM cabinet to be detected through a deep learning model to obtain real features and virtual features; A generating unit, used for generating a candidate region of an ATM cabinet to be detected through a region candidate network; A feature fusion unit, used for fusing the candidate area, the real feature and the virtual feature to obtain a feature representation method of the ATM cabinet to be detected; The inspection unit is used to classify and regress the feature representation method of the ATM cabinet to be inspected and the candidate area through a deep learning model to complete the inspection of the ATM cabinet.
7. The device according to claim 6, characterized in that The acquisition unit comprises: An acquisition module is used to acquire a real image of the ATM cabinet to be inspected through a preset acquisition device; Create a module for creating a virtual environment; The rendering module is used to perform rendering operation on the ATM cabinet to be detected through a rendering engine in the virtual environment to obtain a virtual image corresponding to the ATM cabinet to be detected.
8. The device according to claim 6, characterized in that The generating unit comprises: A feature extraction module is used to extract features from the real image of the ATM cabinet to be detected through a region candidate network to obtain a candidate region feature map; A generation module, used for generating a set of predefined anchor boxes at each position on the candidate region feature map; The operation module is used to perform classification and regression operations on each anchor box through the region candidate network to obtain the initial candidate region; The removal module is used to remove redundant candidate regions from the initial candidate regions through non-maximum suppression to obtain candidate regions.
9. The device according to claim 6, characterized in that The feature fusion unit comprises: A construction module, used to construct a category prototype corresponding to a real feature and a category prototype corresponding to a virtual feature; An establishing module, used for establishing consistency constraints between the category prototype corresponding to the real feature and the category prototype corresponding to the virtual feature; The feature fusion module is used to generate a feature vector corresponding to a candidate box through a regional candidate network under the condition of the consistency constraint, and to fuse the feature vector corresponding to the candidate box, the real feature and the virtual feature to obtain a feature representation method of the ATM cabinet to be detected.
10. The device according to claim 6, characterized in that The inspection unit comprises: An integration module, used to integrate the feature representation method of the ATM cabinet to be detected and the candidate area through a fully connected layer in a deep learning model to obtain an integrated feature vector; A product module, used for multiplying the integrated feature vector and the weight matrix through a fully connected layer in a deep learning model to obtain a product result; The inspection module is used to output the product result through an activation function through a fully connected layer in a deep learning model to achieve classification and position regression, thereby completing the inspection of the ATM cabinet.