Access control system control method, device, equipment, medium and program product

Abstract

This application provides an access control system control method, device, equipment, medium, and program product, relating to the field of artificial intelligence. The method includes: acquiring a target image; extracting shallow edge features and deep semantic features of the target image based on a fully convolutional network, and segmenting the target image based on the fusion result of the shallow edge features and deep semantic features to obtain a target region image; extracting multi-scale features of the target region image to obtain a multi-scale feature map; performing face and vehicle detection on the multi-scale feature map based on preset anchor box parameters to obtain a target detection result; if the target corresponding to the target detection result is an authorized object, generating an access control system opening command. This application segments the target image and extracts key region images, no longer limited by the location of people or vehicles. Simultaneously, it performs face recognition and vehicle detection based on multi-scale features, making the functionality more comprehensive and adaptable to more scenarios.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and in particular to a method, device, equipment, medium, and program product for access control system. Background Technology

[0002] Access control systems are security facilities used to manage the access rights of personnel or vehicles entering and exiting entrances and exits. In areas with high security control requirements, such as the internal areas of banks, access control systems are particularly critical. They not only need to ensure that compliant personnel and authorized vehicles can pass through efficiently, but also need to establish strict access barriers to completely prevent irrelevant personnel and unauthorized vehicles from entering, thereby ensuring the security of important documents, core equipment and other critical assets within the area.

[0003] Currently, traditional access control systems require security personnel to visually identify individuals and vehicles entering and exiting, or rely on basic image processing technology for license plate recognition or facial recognition. However, visual identification is labor-intensive and prone to errors due to fatigue or distraction; relying on basic image processing technology struggles to cope with environmental factors affecting recognition results, leading to significant fluctuations in accuracy. Furthermore, this method requires restrictions on the position of individuals or vehicles, such as requiring individuals to stand directly in front of the facial recognition device. Any deviation in position or angle may cause recognition failure, impacting both access efficiency and operational complexity. Summary of the Invention

[0004] This application provides a method, device, equipment, medium, and program product for access control systems. By fusing shallow edge features and deep semantic features, the target image is segmented, which can accurately locate key areas in the target image and extract the target area image. This breaks through the limitations of access control systems on the location of personnel or vehicles, and then performs face or vehicle detection in key areas. It can achieve efficient, accurate, and multifunctional recognition, and improve the applicability and flexibility of access control systems in complex environments.

[0005] Firstly, this application provides an access control system control method, including:

[0006] Acquire the target image;

[0007] The shallow edge features and deep semantic features of the target image are extracted based on a fully convolutional network, and the target image is segmented based on the fusion result of the shallow edge features and deep semantic features to obtain the target region image.

[0008] Multi-scale features are extracted from the target region image to obtain a multi-scale feature map;

[0009] Based on preset anchor box parameters, face and vehicle detection are performed on multi-scale feature maps to obtain target detection results; the anchor box parameters are obtained through clustering and are used to describe anchor boxes of various sizes for target detection.

[0010] If the target detected is an authorized object, an access control system opening command is generated.

[0011] Secondly, this application provides an access control system control device, comprising:

[0012] The image acquisition module is used to acquire the target image;

[0013] The image segmentation module is used to extract shallow edge features and deep semantic features of the target image based on a fully convolutional network, and to segment the target image based on the fusion result of shallow edge features and deep semantic features to obtain the target region image;

[0014] The feature map extraction module is used to extract multi-scale features from the target region image to obtain a multi-scale feature map.

[0015] The detection result determination module is used to perform face and vehicle detection on multi-scale feature maps based on preset anchor box parameters to obtain target detection results; the anchor box parameters are obtained through clustering and are used to describe anchor boxes of various sizes for target detection.

[0016] The recognition module is used to generate an access control system opening command if the target corresponding to the target detection result is an authorized object.

[0017] Thirdly, this application provides an electronic device, including: a processor, and a memory communicatively connected to the processor;

[0018] The memory stores the instructions that the computer executes;

[0019] The processor executes computer-executable instructions stored in memory to implement the method provided in the first aspect above.

[0020] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the method provided in the first aspect above.

[0021] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the method provided in the first aspect above.

[0022] The access control system control method, device, equipment, medium, and program products provided in this application use a fully convolutional network to extract shallow edge features and deep semantic features from images acquired by the access control system. The results, which fuse shallow and deep semantic features, are used to achieve image segmentation, obtaining key target region images. By fusing shallow and deep features, the computer can first grasp surface information and then analyze internal details; this fusion significantly improves the accuracy and robustness of image analysis. After obtaining the target region image, multi-scale feature extraction is performed only on the target region image, and face recognition and vehicle detection are performed within preset anchor bounding boxes to obtain target detection results. The target detection results are more accurate, and a single method can achieve both face recognition and vehicle detection, improving the applicability and flexibility of this approach, making it suitable for use in complex scenarios. Finally, based on the target corresponding to the target detection results, the access control system determines whether to allow the target to pass, achieving access control system control without restricting the target's location. Attached Figure Description

[0023] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0024] Figure 1 This is a schematic diagram of the structure of an access control system provided in an embodiment of this application;

[0025] Figure 2 A flowchart illustrating an access control system control method provided in an embodiment of this application;

[0026] Figure 3 A schematic diagram of the framework of a target detection model provided in an embodiment of this application;

[0027] Figure 4 A flowchart illustrating another access control system control method provided in this application embodiment;

[0028] Figure 5 A schematic diagram illustrating the training process of the target recognition model provided in this application embodiment;

[0029] Figure 6 A schematic diagram of the structure of an access control system control device provided in an embodiment of this application;

[0030] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0031] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0032] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0033] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of the relevant data all comply with the relevant laws, regulations, and standards of the relevant countries and regions, have taken necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation access points for users to choose to authorize or refuse.

[0034] Furthermore, the technical solution involved in this application, which involves big data analysis of user information (including but not limited to personal biometrics, identity data, consumption data, asset data, electronic terminal operation data, etc.) and the use of artificial intelligence technology for automated decision-making, and makes decisions that have a significant impact on personal rights based on the results of automated decision-making, provides users with corresponding operation entry points for users to choose to agree to or reject the results of automated decision-making; if the user chooses to reject, the process will proceed to the expert decision-making process.

[0035] It should be noted that the access control system control methods, devices, equipment, media and program products provided in this application can be used in the field of artificial intelligence, or in any field other than artificial intelligence. The application fields of the access control system control methods, devices, equipment, media and program products in this application are not limited.

[0036] Figure 1 This is a schematic diagram of an access control system provided in an embodiment of this application. Figure 1 As shown, access control systems typically deploy data acquisition devices (such as...). Figure 1The system includes card readers and cameras, barrier gates, and pedestrian access doors. When a vehicle or person intends to enter the area controlled by the access control system, the identification device determines whether to raise the barrier gate or open the pedestrian access door by identifying whether the vehicle, person, or card is authorized. This application is applicable to scenarios where access control systems manage the entry and exit of vehicles and personnel, and can specifically be used to control access control systems.

[0037] In scenarios with high security control requirements, such as the internal areas of banks, the level of control over personnel entering and exiting is even higher, placing greater demands on the access control systems. These systems need to ensure efficient passage for compliant individuals such as staff and authorized vehicles, while simultaneously improving identification accuracy and strictly preventing unauthorized personnel and vehicles from entering.

[0038] Currently, some access control systems rely on security personnel manually identifying staff or authorized vehicles through video streams from surveillance cameras, or triggering gates or pedestrian access doors upon receiving a successful card swipe from a card reader. This method is labor-intensive and prone to errors due to fatigue or distraction, allowing unauthorized personnel or vehicles to enter the controlled area.

[0039] With the development of identification and automatic control technologies, some access control systems can now directly identify whether an individual or vehicle is authorized, automatically raising the barrier or opening the pedestrian access door to allow passage. Specifically, these systems can grant access after the card reader displays a successful card swipe, and can also rely on basic image processing technology to perform license plate or facial recognition, allowing passage when the individual or vehicle is identified as authorized.

[0040] However, using card-swiping devices requires people or vehicles to be located in a fixed area, and basic image recognition technology also has certain limitations on the position of people or vehicles. If the position of people or vehicles shifts, recognition may fail, affecting passage efficiency and increasing the operational threshold. In addition, in basic image recognition technology, the background environment in the image has a significant impact on recognition, resulting in large fluctuations in recognition accuracy and low recognition stability.

[0041] The access control system control method provided in this application aims to solve the above-mentioned technical problems. Specifically, it uses a fully convolutional network to extract shallow edge features and deep semantic features from an image, and fuses features at different levels. Based on the fusion result, it segments the key recognition regions in the image to obtain the target region image. Regardless of the target region's location in the image, the target region image can be effectively obtained. Then, multi-scale feature extraction is performed on the target region image, and face recognition and vehicle detection are performed within preset anchor bounding boxes, resulting in more accurate detection results. Finally, based on whether the detected object is an authorized user, the access control system is controlled to open, such as raising the barrier gate or opening the pedestrian access door. This achieves automatic access control system control without manual operation, resulting in more accurate control. At the same time, it avoids the positional limitations of basic image recognition technologies, improving the applicability and flexibility of this method.

[0042] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0043] Figure 2 This is a flowchart illustrating an access control system control method provided in an embodiment of this application. The access control system control method provided in this application can be executed by an electronic device with corresponding processing capabilities. The electronic device includes, but is not limited to, the control unit of the access control system, and electronic devices communicatively connected to the access control system. Figure 2 As shown, the method provided in this embodiment includes the following steps:

[0044] Step S201: Obtain the target image.

[0045] The target image is an image that includes the objects to be identified. The objects to be identified include people and vehicles.

[0046] In this step, the electronic device can communicate with the camera of the access control system to acquire target images of people or vehicles to be identified, including those to be entered into the area controlled by the access control system.

[0047] In some embodiments, after acquiring the target image, the target image may be preprocessed. Preprocessing includes, but is not limited to, image enhancement and image normalization.

[0048] For example, image normalization can be performed by proportionally normalizing the resolution of the target image based on the length of its shorter side, such as adjusting it to a size of 500×500.

[0049] Since the hardware resources of electronic devices limit computing resources, and the target images obtained from each execution of the method are of different sizes, it will increase the difficulty of subsequent processing. By normalizing the image, the target images obtained from each execution are all in one dimension, which solves the above problems, that is, unifies the standard and reduces the subsequent computing burden.

[0050] Step S202: Extract shallow edge features and deep semantic features of the target image based on a fully convolutional network, and segment the target image based on the fusion result of shallow edge features and deep semantic features to obtain the target region image.

[0051] Fully convolutional networks are used to extract features from target images.

[0052] The fully convolutional network in this application is a skip-structure-based fully convolutional network. The fully convolutional network includes upsampling layers and downsampling layers. The downsampling layers are used to gradually reduce the size of the target image through convolutional layers and pooling layers, capturing the shallow edge features and deep semantic features of the target image; the upsampling layers are used to amplify the features obtained by downsampling through deconvolution or interpolation, gradually restoring the original image size, and obtaining the fusion result of shallow edge features and deep semantic features.

[0053] For example, the downsampling layer includes convolutional modules C1-C8, where convolutional modules C1-C5 extract shallow edge features and convolutional modules C6-C8 extract deep semantic features.

[0054] In this step, the fully convolutional network provided in this application downsamples the target image by stacking convolutional kernels, extracting shallow edge features and deep semantic features. The deep semantic features are then fused with the shallow edge features through multiple upsampling operations to obtain a fused result. The target image is then segmented based on the fused result to obtain the target region image.

[0055] For example, a fully convolutional network can input the target image into a downsampling layer to obtain shallow edge features and deep semantic features. Then, the deep semantic features are upsampled by 2 times and fused with the shallow edge features output from the pooling layer. Finally, the fused result is upsampled by 16 times to obtain the final fused result.

[0056] For example, deep semantic features and shallow edge features from the pooling layer can be fused together, and the fused result can be upsampled by 8 times to obtain the fused result.

[0057] To segment the target image based on the fusion result, the fusion result can be binarized at the pixel level, where 1 represents the target region and 0 represents the background. The binarized result is then logically ANDed with the target image to obtain the target region image after removing the background region.

[0058] The target image can be segmented based on the fusion result, or the fusion result can be binarized at the pixel level, where 1 represents the target region and 0 represents the background. The region corresponding to 1 is segmented from the target image to obtain the target region image.

[0059] By obtaining the target region image, high-quality image data without background interference is provided for subsequent image processing operations. At the same time, by fusing shallow edge features and deep semantic features, the overall accuracy of the image is improved by using details to complete the image, further ensuring that the background environment is completely segmented, and the segmented target region image is more accurate and complete.

[0060] Step S203: Extract multi-scale features from the target region image to obtain a multi-scale feature map.

[0061] Multi-scale feature maps are feature maps of different resolutions, each adapted to objects of different sizes.

[0062] For example, multi-scale feature maps may include high-resolution feature maps, medium-resolution feature maps, and low-resolution feature maps.

[0063] High-resolution feature maps, such as 52×52, preserve local details and are better suited for small target detection, i.e., long-range detection; medium-resolution feature maps, such as 26×26, balance details and semantics and are better suited for medium target detection, i.e., medium-range detection; low-resolution feature maps, such as 13×13, capture global semantics and are suitable for large target detection, i.e., short-range detection.

[0064] In this step, the target region image can be input into the feature extraction network. The feature extraction network extracts multi-scale feature maps of the target region image through a cascaded process, resulting in multi-scale feature maps.

[0065] The feature extraction network provided in this application uses convolutional layers with a stride of 2 instead of pooling layers to avoid gradient vanishing, and deepens the network through residual structures. These residual structures include batch normalization and activation functions.

[0066] Specifically, multiple basic units of convolutional layer-residual layer-downsampling layer are constructed and connected in series according to a preset logic (the last unit can omit the downsampling layer). That is, the output feature of the previous unit is the input feature of the next unit. The output features of each unit are obtained to complete the progressive extraction of low-level features-medium-high-level features, that is, to obtain size maps of 52×52, 26×26 and 13×13 respectively.

[0067] In some embodiments, in order to make full use of feature information, feature maps of size 13×13 and 26×26 in the multi-scale feature map can be upsampled by 2 times, and tensor splicing of feature maps of different sizes can be performed by drawing on the idea of ​​feature pyramid. Finally, regression prediction of target category and shape can be performed independently on feature maps of three scales to obtain target detection results.

[0068] Step S204: Based on the preset anchor box parameters, perform face and vehicle detection on the multi-scale feature map to obtain the target detection results.

[0069] Anchor boxes are pre-defined rectangular or other shaped boxes marked on target objects after they have been identified. Anchor box parameters include the dimensions of the anchor box, such as length and width. These parameters are obtained through clustering and are used to describe anchor boxes of various sizes for target detection. In other words, anchor box parameters are designed to accommodate targets of different sizes and shapes.

[0070] For different types of target objects, such as people or vehicles, the space they occupy on the target image is not consistent. If anchor boxes are generated with fixed anchor box parameters, the anchor boxes will not be able to accurately mark the target objects, and may even obscure the information of the target objects.

[0071] To address this issue, this application determines the anchor box parameters of the target object by comparing the results of multi-scale feature maps and clustering.

[0072] Specifically, the similarity between the multi-scale feature map and multiple preset cluster centers is calculated, and the anchor box parameters corresponding to the feature clusters of the cluster centers that meet the similarity conditions are selected as the anchor box parameters of the target object.

[0073] The anchor box parameters corresponding to the feature clusters are pre-set. Specifically, multi-scale feature maps corresponding to multiple historical target images are pre-acquired. These multi-scale feature maps are clustered to obtain multiple feature clusters and their cluster centers. Anchor box parameters are then set for each feature cluster. For example, anchor boxes can be manually labeled for the multi-scale feature maps corresponding to multiple historical target images, or a recognition algorithm can be used to label them. The anchor box parameters corresponding to the feature cluster are determined based on the statistical values, such as the average value, of the anchor box parameters for each historical target image within a feature cluster.

[0074] For example, the K-means++ algorithm can be used to cluster the target image, classifying it into categories such as people and vehicles. For the people category, anchor box parameters are set for the corresponding people; for the vehicles category, anchor box parameters are set for the corresponding vehicles.

[0075] The target detection results include the target, the anchor bounding box labeled on the target, and the target category.

[0076] In this step, the determined anchor box parameters and multi-scale feature maps are input into a pre-trained target detection model. The target detection model performs face and vehicle detection on the multi-scale feature maps to obtain the target detection results.

[0077] The pre-trained object detection model is a single-stage real-time object detection model based on a fully convolutional network architecture. This object detection model does not require the generation of candidate regions and can directly predict anchor boxes on multi-scale feature maps through the fully convolutional network.

[0078] Specifically, the object detection model divides the input multi-scale feature map into an S×S grid (the value of S varies for different scales of feature maps). Based on the preset anchor box parameters, it determines the anchor box corresponding to each grid. The tensor size can be expressed as S×S×[3×(X+C)], where X includes the coordinate information (x, y, w, h) and confidence of the anchor box, and C represents the number of predicted categories. Finally, through threshold setting and non-maximum suppression, the anchor box with the highest confidence is output.

[0079] For example, Figure 3 This is a schematic diagram illustrating the framework of a target detection model provided in an embodiment of this application. Figure 3 As shown, the target detection model also includes a feature extraction network, which is used to extract multi-scale features from the target region image to obtain a multi-scale feature map, i.e. Figure 3 Then, based on the preset anchor box parameters, face and vehicle detection are performed on the multi-scale feature map through upsampling, resulting in... Figure 3 The target detection results in the middle.

[0080] Step S205: If the target corresponding to the target detection result is an authorized object, then generate an access control system opening command.

[0081] In this step, it is determined whether the target corresponding to the target detection result is an authorized object, such as whether the license plate is an authorized license plate or the person is an authorized person. If the target is an authorized object, an access control system opening command is generated, which can control the barrier gate to lift or control the pedestrian passage door to open.

[0082] Optionally, the method further includes: if the target corresponding to the target detection result is an unauthorized object, generating a prohibition command for the access control system, and performing at least one of the following operations: storing and sending the target detection result and its corresponding target image; triggering a preset alarm mechanism, the preset alarm mechanism including at least one of the following: controlling the flashing of warning lights, broadcasting preset voice prompts, and pushing alarm information to the gatekeeper terminal.

[0083] In this embodiment, if the target detected is not an authorized object, the person or vehicle is prohibited from entering the area, and a prohibition command is generated for the access control system, prohibiting the gate arm from being raised or the pedestrian passage door from being opened.

[0084] After generating the command to prohibit opening, the target detection result and its corresponding target image are stored and sent to the control terminal, and / or a preset alarm mechanism is triggered.

[0085] The preset alarm mechanism includes, but is not limited to, controlling the flashing of warning lights, broadcasting preset voice prompts, and pushing alarm information to the gatekeeper terminal.

[0086] By generating a prohibition command and triggering an alarm mechanism when an unauthorized object is detected, the functionality of the access control system is further improved, making the access control system more reliable.

[0087] The access control system control method provided in this application uses a fully convolutional network to extract shallow edge features and deep semantic features from images acquired by the access control system. It then uses the fusion of shallow and deep semantic features to achieve image segmentation, obtaining key target region images. By fusing shallow and deep features, the computer can first grasp surface information and then analyze internal details; this fusion significantly improves the accuracy and robustness of image analysis. After obtaining the target region image, multi-scale feature extraction is performed only on the target region image, and face recognition and vehicle detection are performed within preset anchor bounding boxes to obtain target detection results. The target detection results are more accurate, and a single method can achieve both face recognition and vehicle detection, improving the applicability and flexibility of the method, making it suitable for use in complex scenarios. Finally, based on the target corresponding to the target detection results, the access control system determines whether to allow the target to pass, achieving access control system control without restricting the target's location.

[0088] In one possible implementation, the anchor box parameters corresponding to the feature clusters can be obtained through training.

[0089] Specifically, first, a preset number of fixed parameters is set. The preset number is a configurable parameter. Increasing the preset number will reduce the calculation speed and increase the calculation cost. When the fixed number is 9, the calculation cost and effect are relatively balanced. The preferred preset number is 9.

[0090] Then, images including the facial information of authorized personnel and images including authorized vehicles, stored in history, are acquired and used as training samples. Ground truth anchor boxes are obtained from the training samples, and training anchor boxes are generated for the training samples using the method provided in the above embodiments. The anchor box parameters of the training anchor boxes can be initial values ​​corresponding to each feature cluster.

[0091] Next, the training samples are clustered using the K-means or K-means++ algorithm to obtain multiple feature clusters. The intersection-union ratio (IU) of the ground truth anchor boxes and the training anchor boxes for each feature cluster is calculated, along with the average loss between the training and ground truth anchor boxes. Based on the average loss, the initial values ​​corresponding to the feature clusters are adjusted.

[0092] The average loss can be expressed as:

[0093]

[0094] Where D is the average loss; B represents the ground truth anchor box; C represents the cluster center; n is the total number of samples; k is the number of cluster centers, i.e., the number of feature clusters; n k I is the number of training samples in the k-th feature cluster; IOU (B, C) represents the intersection-union ratio of the ground truth anchor boxes and the training anchor boxes.

[0095] Figure 4 This is a flowchart illustrating another access control system control method provided in this application embodiment. The method provided in this embodiment is... Figure 2 Based on the illustrated embodiment, steps S201, S203, and S204 have been further refined. For example... Figure 4 As shown, the method provided in this embodiment includes:

[0096] Step S401: Acquire target images collected by multiple data acquisition devices.

[0097] Multiple data acquisition devices include visible light cameras and infrared cameras.

[0098] Data acquisition equipment refers to image acquisition devices deployed in the environment surrounding the access control system. This includes, but is not limited to, visible light cameras and infrared cameras.

[0099] Infrared cameras enable access control systems to accurately and reliably manage people and vehicles passing through, both day and night. At the same time, multiple data acquisition devices collect target images, enabling multi-directional and multi-angle data collection of people or vehicles, laying a reliable data foundation for subsequent processing.

[0100] Step S402: Extract shallow edge features and deep semantic features of the target image based on a fully convolutional network, and segment the target image based on the fusion result of shallow edge features and deep semantic features to obtain the target region image.

[0101] Step S403: Extract features from the target region image based on a deep residual network to generate feature maps at least three scales.

[0102] The deep residual network provided in this embodiment uses convolutional compression of channels, convolutional feature extraction, and skip connections as its basic residual structure, achieving deep feature extraction through multiple rounds of stacking. This deep residual network also includes batch normalization (DN) and activation functions. Batch normalization and activation functions further avoid the gradient vanishing problem.

[0103] Batch normalization is designed to make training more stable and accelerate network convergence. Its calculation formula is as follows:

[0104]

[0105] Where x is the batch normalized input and y is the result of the basic residual structure output; y is the batch normalized output; and γ and β are learnable parameters.

[0106] The activation function can be a non-saturated LeakyReLU activation function. When the input value of the activation function is negative, a small slope is given to it, which can reduce the occurrence of silent units and allow neurons to continue learning. Specifically, it can be expressed as:

[0107]

[0108] Among them, a i This represents the slope of each input value x, which is a pre-set configurable parameter.

[0109] In this step, the target region image is input into a deep residual network. Each stage of the deep residual network sequentially extracts features from the target region image, generating feature maps at least three scales, such as 52×52, 26×26, and 13×13 feature maps.

[0110] Step S404: Upsample and fuse the feature maps at at least three scales and perform skip connections to obtain multi-scale feature maps.

[0111] In this step, upsampling is performed on the generated feature maps at least three scales, such as 2x upsampling, and skip connections are made between feature maps of different sizes to obtain multi-scale feature maps.

[0112] Step S405: For each scale of the feature map in the multi-scale feature map, assign anchor point bounding box parameters to the feature map of that scale.

[0113] Anchor box parameters include anchor box parameters at multiple scales. For example, assuming the anchor box parameters include length and width, the anchor box parameters specifically include the length and width of the anchor box for each scale feature map.

[0114] In this step, for each scale of the feature map in the multi-scale feature map, the corresponding anchor box parameters are assigned to the feature map of that scale according to the cluster center corresponding to that feature map.

[0115] Step S406: On the feature map at this scale, perform bounding box regression and class prediction based on the assigned anchor box parameters to output candidate detection boxes for faces and vehicles.

[0116] Bounding box regression is a technique used to precisely adjust the position and size of preset anchor point boxes.

[0117] In this step, initial anchor boxes are generated for the feature map at this scale based on the assigned anchor box parameters. Then, bounding box regression is used to predict the offset of the initial anchor boxes from the ideal anchor boxes, and the initial anchor boxes are adjusted towards the ideal anchor boxes to obtain multiple candidate detection boxes. A class prediction-based model is then used to predict the category of the target contained in each candidate detection box, i.e., whether the candidate detection box corresponds to a face or a vehicle, and the probability of belonging to that category.

[0118] Step S407: Select the target detection box from each candidate detection box corresponding to the multi-scale feature map.

[0119] In this step, the candidate detection box with the highest confidence among the candidate detection boxes corresponding to the multi-scale feature map is determined as the target detection box.

[0120] In some embodiments, candidate detection boxes with a confidence level greater than or equal to a preset threshold can be selected from each candidate detection box corresponding to the multi-scale feature map as target detection boxes.

[0121] In some embodiments, before selecting the target detection box from each candidate detection box corresponding to the multi-scale feature map, a threshold setting and non-maximum suppression can be applied to the candidate detection boxes to select accurate and non-repeating candidate detection boxes.

[0122] Step S408: Extract the target feature vector from the target image based on the target detection box.

[0123] In this step, a target feature vector is extracted based on the features of the target image corresponding to the target detection box. This target feature vector is a biometric vector that can uniquely represent a specific person, or a vector representing the features of a vehicle.

[0124] Specifically, deep convolution and attention mechanisms can be used to extract core, fixed-dimensional target feature vectors from the features of the target image corresponding to the target detection box.

[0125] Step S409: Output the target detection result, which includes the category of the target detection box and the target feature vector.

[0126] In this step, the output includes the category of the object detection box and the object feature vector that can uniquely represent the object.

[0127] Optionally, the method provided in this embodiment further includes: filtering authorized objects in the authorized feature library based on the category of the target detection box in the target detection result to obtain multiple candidate objects; calculating the similarity between the target feature vector in the target detection result and the object feature vector corresponding to each candidate object; if there is a candidate object with a similarity greater than a preset threshold, then determining the target corresponding to the target detection result as an authorized object.

[0128] The authorized feature library is a database that stores the feature vectors of each authorized object. Specifically, images of all authorized objects (authorized personnel and authorized vehicles) can be acquired, and the feature vectors of each authorized object can be obtained using the methods described above. These feature vectors are then stored in the authorized feature library.

[0129] To avoid the need to compare the target feature vector with a large number of object feature vectors during recognition, which consumes a lot of computing resources, the object feature vectors in the authorized feature library can be classified according to the category of the authorized object, such as people or vehicles.

[0130] Furthermore, during identification, based on the category of the target detection box in the target detection result, authorized objects of that category in the authorized feature library can be filtered to obtain multiple candidate objects. Then, the object feature vectors corresponding to the multiple candidate objects of that category are retrieved, and their similarity is calculated with the target feature vector in the target detection result. If there is a candidate object with a similarity greater than a preset threshold (e.g., 0.9), the target corresponding to the target detection result is determined as the authorized object.

[0131] By selecting candidate objects and conducting comparative analysis among them, the need for comparison analysis with a large number of authorized objects is avoided, which significantly reduces the amount of data processing required, reduces the computing burden on computers, and improves resource utilization.

[0132] Step S410: If the target corresponding to the target detection result is an authorized object, then generate an access control system opening command.

[0133] Optionally, after generating the access control system's opening command, the method further includes: storing the target detection results as new samples in the fine-tuning sample set; updating the authorized feature library based on the samples in the fine-tuning sample set and the clustering algorithm when the number of samples in the fine-tuning sample set reaches a preset number, or when an update command is received.

[0134] The fine-tuning sample set stores fine-tuning samples. These fine-tuning samples are used to adjust the authorized feature library.

[0135] In this step, the target and its feature vector corresponding to the target detection result are added as new samples and stored in the fine-tuning sample set. When the number of samples in the fine-tuning sample set reaches a preset number, or when an update instruction is received, the object feature vectors of the authorized objects corresponding to the samples in the authorized feature library are adjusted.

[0136] Specifically, based on samples of the same authorized object in the fine-tuning sample set and the object feature vector of that authorized object in the authorization feature library, a clustering algorithm is used to perform clustering, obtaining the feature vector corresponding to the cluster center. Based on the feature vector corresponding to the cluster center, the object feature vector of the corresponding authorized object in the authorization feature library is updated.

[0137] By adjusting the authorized feature library using samples from the fine-tuning sample set, the object feature vectors in the feature database can always remain accurate and timely, thereby achieving efficient recognition.

[0138] In this embodiment, multi-scale features are processed through upsampling fusion and skip connections, so that the final multi-scale feature map has both high-resolution details and deep semantic information. This fundamentally solves the problem of lost details or insufficient semantics in single-scale features and improves the quality of multi-scale feature maps. At the same time, by first generating detection boxes and then extracting feature vectors based on the detection boxes, the extracted feature vectors are purer and less affected by other factors, while accurately focusing on the target region. This avoids redundant information in the feature vectors caused by full-image extraction. Furthermore, the preset anchor point parameters are adjusted when generating detection boxes, making the final generated detection boxes more accurate and closely match the actual target.

[0139] In one possible implementation, step S201, acquiring the target image, may include: acquiring raw images acquired by multiple data acquisition devices; the multiple data acquisition devices include a visible light camera and an infrared camera; determining the weight of the raw images acquired by each of the multiple data acquisition devices based on the data acquired by the ambient light sensor; and performing weighted fusion of the raw images acquired by the multiple data acquisition devices based on the weight of each raw image to obtain the target image.

[0140] An ambient light sensor is used to collect light intensity data of the surrounding environment of the data acquisition device.

[0141] In this step, raw images are acquired from multiple data acquisition devices, including raw images from a visible light camera and raw images from an infrared camera. Based on the data acquired by the ambient light sensor, the weights of the raw images acquired by each device are determined. Based on these weights, the pixel values ​​at corresponding locations in the raw images from the multiple data acquisition devices are weighted and summed, or the raw images from the multiple data acquisition devices are multiplied by their weights and then stitched together to obtain the target image.

[0142] Based on the data collected by the ambient light sensor, the weight of the raw images collected by each of the multiple data acquisition devices is determined. This can be achieved by: increasing the weight of the raw image collected by the visible light camera (e.g., 0.6) and decreasing the weight of the raw image collected by the infrared camera (e.g., 0.4) if the data indicates that the ambient light intensity is less than the preset brightness, decreasing the weight of the raw image collected by the visible light camera (e.g., 0.4) and increasing the weight of the raw image collected by the infrared camera (e.g., 0.6).

[0143] By weighted fusion of raw data from multiple data acquisition devices, high-quality target images can be obtained under any lighting conditions, laying a high-quality data foundation for subsequent processing.

[0144] In one possible implementation, after acquiring the target image, the access control system control method provided in this application can be implemented by a pre-trained target recognition model.

[0145] Specifically, the access control system control methods can be as follows:

[0146] Acquire the target image.

[0147] The target image is input into a pre-trained target recognition model. The target recognition model extracts shallow edge features and deep semantic features of the target image based on a fully convolutional network, and segments the target image based on the fusion result of shallow edge features and deep semantic features to obtain target region images. Multi-scale features of the target region images are extracted to obtain multi-scale feature maps. Based on preset anchor box parameters, face and vehicle detection are performed on the multi-scale feature maps to obtain target detection results. The anchor box parameters are obtained through clustering and are used to describe anchor boxes of various sizes for target detection.

[0148] If the target detected is an authorized object, an access control system opening command is generated.

[0149] To implement this method, the target recognition model can be based on the implementation principles provided in the above embodiments, including an image segmentation layer and a target recognition layer. The image segmentation layer further includes a feature extraction network and a segmentation network. The target recognition layer further includes a feature extraction network and a recognition network.

[0150] The image segmentation layer and the object recognition layer are trained so that they can ultimately obtain the object detection result.

[0151] Figure 5 This is a schematic diagram illustrating the training process of the target recognition model provided in an embodiment of this application. Figure 5 As shown, historically stored images including the facial information of authorized personnel and images including authorized vehicles are obtained and used as training and testing samples. A labeling tool is used to set corresponding labels for each training and testing sample, such as labels set as category, specific personnel name or license plate number, etc.

[0152] The image segmentation layer is trained using training samples, and the parameters in the image segmentation layer are adjusted. For example, the training samples are first passed sequentially through the feature extraction network and the segmentation network of the image segmentation layer to obtain the fusion result of the training samples based on shallow edge features and deep semantic features. The parameters of the image segmentation layer are then adjusted based on the fusion result of the training samples.

[0153] After training with the training samples, an image segmentation layer is constructed based on the adjusted parameters. The image segmentation layer is then validated using test samples.

[0154] After verification, the test sample is processed through the image segmentation layer to obtain the target region image of the test sample. This target region image of the test sample is used as both the training and test sample for the target recognition layer.

[0155] The target recognition layer is trained using training samples, and its parameters are adjusted to obtain a weight model. The target recognition layer is then constructed based on this weight model. Next, test samples are input into the constructed target recognition layer to obtain target detection results. Finally, the loss value between the obtained target detection results and the actual target detection results is used to determine whether the target recognition layer has been successfully trained.

[0156] The loss value can be expressed as:

[0157]

[0158] in, represents the x and y coordinates of the center point of the target detection box in the i-th grid, and the width and height of the target detection box, respectively; S1 and S2 represent the x and y coordinates of the center point of the ground truth detection box in the i-th grid, and the width and height of the ground truth detection box, respectively; S2 is the number of image segmentation grids, and B is each detection box; Ci and The predicted category and the actual category are compared; and λ represents the confidence scores of the obtained category and the actual category, respectively. coord , λ noobj For loss weights; , This indicates whether the j-th detection box in the i-th grid corresponds to the target. If they correspond, the result is 1; otherwise, the result is 0.

[0159] Figure 6 This is a schematic diagram of the structure of an access control system control device provided in an embodiment of this application. Figure 6 As shown, the access control system control device provided in this embodiment includes an image acquisition module 601, an image segmentation module 602, a feature map extraction module 603, a detection result determination module 604, and a recognition module 605.

[0160] Image acquisition module 601 is used to acquire target images; image segmentation module 602 is used to extract shallow edge features and deep semantic features of the target image based on a fully convolutional network, and to segment the target image based on the fusion result of shallow edge features and deep semantic features to obtain target region images; feature map extraction module 603 is used to extract multi-scale features of the target region image to obtain multi-scale feature maps; detection result determination module 604 is used to perform face and vehicle detection on the multi-scale feature maps based on preset anchor box parameters to obtain target detection results; the anchor box parameters are obtained through clustering and are used to describe anchor boxes of various sizes for target detection; recognition module 605 is used to generate an access control system opening command if the target corresponding to the target detection result is an authorized object.

[0161] Optionally, the feature map extraction module 603 is specifically used for:

[0162] Feature extraction is performed on the target region image based on a deep residual network to generate feature maps at least three scales; upsampling fusion and skip connections are then performed on the feature maps at least three scales to obtain multi-scale feature maps.

[0163] Optionally, the detection result determination module 604 is specifically used for:

[0164] For each scale of the multi-scale feature map, anchor box parameters are assigned to the feature map at that scale. On the feature map at that scale, bounding box regression and class prediction are performed based on the assigned anchor box parameters, and candidate detection boxes for faces and vehicles are output. Target detection boxes are obtained by filtering from each candidate detection box corresponding to the multi-scale feature map. Based on the target detection boxes, target feature vectors are extracted from the target image. The target detection result is output, where the target detection result includes the class of the target detection box and the target feature vector.

[0165] Optionally, the access control system control device also includes an authorized object determination module, used for:

[0166] Based on the category of the target detection box in the target detection result, the authorized objects in the authorized feature library are filtered to obtain multiple candidate objects; the similarity between the target feature vector in the target detection result and the object feature vector corresponding to each candidate object is calculated; if there is a candidate object with a similarity greater than a preset threshold, the target corresponding to the target detection result is determined to be the authorized object.

[0167] Optionally, the access control system control device also includes a sample fine-tuning module for:

[0168] After generating the access control system's opening command, the target detection results are stored as new samples in the fine-tuning sample set. When the number of samples in the fine-tuning sample set reaches a preset number, or when an update command is received, the authorized feature library is updated based on the samples in the fine-tuning sample set and the clustering algorithm.

[0169] Optionally, the image acquisition module 601 is specifically used for:

[0170] Acquire target images from multiple data acquisition devices, including visible light cameras and infrared cameras.

[0171] Optionally, the image acquisition module 601 is specifically used for:

[0172] The system acquires raw images from multiple data acquisition devices, including visible light cameras and infrared cameras. Based on data acquired by an ambient light sensor, it determines the weights of the raw images acquired by each device. Based on the weights of each raw image, it performs weighted fusion of the raw images acquired by the multiple data acquisition devices to obtain the target image.

[0173] Optionally, the access control system control device also includes an unauthorized object processing module, used for:

[0174] If the target detected is an unauthorized object, a prohibition command for the access control system is generated, and at least one of the following operations is performed: storing and sending the target detection result and its corresponding target image; triggering a preset alarm mechanism, which includes at least one of the following: controlling the flashing of warning lights, broadcasting preset voice prompts, and pushing alarm information to the gatekeeper terminal.

[0175] The access control system control device provided in this application can be used to execute the technical solution of the access control system control method provided in any of the above embodiments of this application. Its implementation principle and technical effect are similar, and will not be described again here.

[0176] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 7As shown, the electronic device of this embodiment may include: at least one processor 701; and a memory 702 communicatively connected to at least one processor; wherein the memory 702 stores instructions that can be executed by at least one processor 701, and the instructions are executed by at least one processor 701 to cause the electronic device to perform the method as described in any of the above embodiments.

[0177] Optionally, the memory 702 can be either standalone or integrated with the processor 701. When the memory 702 is set up independently, the device also includes a bus for connecting the memory 702 and the processor 701.

[0178] The implementation principle and technical effects of the electronic device provided in this embodiment can be found in the foregoing embodiments, and will not be repeated here.

[0179] This application also provides a computer-readable storage medium storing computer-executable instructions. When the computer-executable instructions are executed by a processor, the methods provided in any of the foregoing embodiments can be implemented.

[0180] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the method provided in any of the foregoing embodiments.

[0181] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0182] It should be further noted that although the steps in the flowchart are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0183] It should be understood that the above-described device embodiments are merely illustrative, and the device of this application can also be implemented in other ways. For example, the division of units / modules in the above embodiments is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units, modules, or components may be combined, or integrated into another system, or some features may be ignored or not executed.

[0184] Furthermore, unless otherwise specified, the functional units / modules in the various embodiments of this application can be integrated into one unit / module, or each unit / module can exist physically separately, or two or more units / modules can be integrated together. The integrated units / modules described above can be implemented in hardware or as software program modules.

[0185] When integrated units / modules are implemented in hardware, the hardware can be digital circuits, analog circuits, etc. The physical implementation of the hardware structure includes, but is not limited to, transistors, memristors, etc. Unless otherwise specified, the processor can be any suitable hardware processor, such as a CPU, GPU, FPGA, DSP, and ASIC, etc. Unless otherwise specified, the storage unit can be any suitable magnetic or magneto-optical storage medium, such as Resistive Random Access Memory (RRAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Enhanced Dynamic Random Access Memory (EDRAM), High-Bandwidth Memory (HBM), Hybrid Memory Cube (HMC), etc.

[0186] If the integrated unit / module is implemented as a software program module and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0187] In the above embodiments, the descriptions of each embodiment have their own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0188] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0189] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method of controlling an access control system, characterized by, include: Acquire the target image; The shallow edge features and deep semantic features of the target image are extracted based on a fully convolutional network, and the target image is segmented based on the fusion result of the shallow edge features and deep semantic features to obtain the target region image; Multi-scale features are extracted from the target region image to obtain a multi-scale feature map; Based on preset anchor box parameters, face and vehicle detection are performed on the multi-scale feature map to obtain target detection results; the anchor box parameters are obtained through clustering and are used to describe anchor boxes of various sizes for target detection. If the target corresponding to the target detection result is an authorized object, then an access control system opening command is generated.

2. The method of claim 1, wherein, The step of extracting multi-scale features from the target region image to obtain a multi-scale feature map includes: Feature extraction is performed on the target region image based on a deep residual network to generate feature maps at least three scales. The feature maps at at least three scales are upsampled, fused, and skipped to obtain the multi-scale feature maps.

3. The method of claim 1, wherein, Based on preset anchor box parameters, face and vehicle detection is performed on the multi-scale feature map to obtain target detection results, including: For each scale of the feature map in the multi-scale feature map, assign anchor point bounding box parameters to the feature map of that scale; On the feature map at this scale, bounding box regression and class prediction are performed based on the assigned anchor box parameters, outputting candidate detection boxes for faces and vehicles; The target detection box is obtained by filtering from each candidate detection box corresponding to the multi-scale feature map; Based on the target detection box, extract the target feature vector from the target image; Output the target detection result, wherein the target detection result includes the category of the target detection box and the target feature vector.

4. The method of claim 3, wherein, The method further includes: Based on the category of the target detection box in the target detection result, the authorized objects in the authorized feature library are filtered to obtain multiple candidate objects; Calculate the similarity between the target feature vector in the target detection result and the object feature vector corresponding to each candidate object; If there are candidate objects with a similarity greater than a preset threshold, then the target corresponding to the target detection result is determined to be an authorized object.

5. The method according to claim 4, characterized in that, After generating the access control system's open command, the method further includes: The target detection results are stored as new samples in the fine-tuning sample set; When the number of samples in the fine-tuning sample set reaches a preset number, or when an update instruction is received, the authorized feature library is updated based on the samples in the fine-tuning sample set and the clustering algorithm.

6. The method according to any one of claims 1-5, characterized in that, The acquisition of the target image includes: Acquire target images from multiple data acquisition devices; the multiple data acquisition devices include visible light cameras and infrared cameras.

7. The method according to any one of claims 1-5, characterized in that, The acquisition of the target image includes: Acquire raw images from multiple data acquisition devices; the multiple data acquisition devices include visible light cameras and infrared cameras; Based on the data collected by the ambient light sensor, the weight of the original image collected by each of the multiple data acquisition devices is determined; Based on the weights of each of the original images, the original images acquired by the multiple data acquisition devices are weighted and fused to obtain the target image.

8. The method according to any one of claims 1-5, characterized in that, The method further includes: If the target detected is an unauthorized object, then a prohibition command for the access control system is generated, and at least one of the following operations is performed: Store and send the target detection results and their corresponding target images; Trigger a preset alarm mechanism, which includes at least one of the following: controlling the warning light to flash, broadcasting a preset voice prompt, and pushing alarm information to the gatekeeper terminal.

9. A control device for an access control system, characterized in that, include: The image acquisition module is used to acquire the target image; The image segmentation module is used to extract shallow edge features and deep semantic features of the target image based on a fully convolutional network, and to segment the target image based on the fusion result of the shallow edge features and deep semantic features to obtain a target region image; The feature map extraction module is used to extract multi-scale features from the target region image to obtain a multi-scale feature map; The detection result determination module is used to perform face and vehicle detection on the multi-scale feature map based on preset anchor box parameters to obtain target detection results; the anchor box parameters are obtained through clustering and are used to describe anchor boxes of various sizes for target detection. The identification module is used to generate an access control system opening command if the target corresponding to the target detection result is an authorized object.

10. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 8.

12. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method of any one of claims 1 to 8.

Images