Green base station energy saving method based on self-attention and convolutional deep learning fusion

Abstract

The invention discloses a green base station energy-saving method based on self-attention and convolutional deep learning fusion, and solves the problems that base station resources are allocated through figure number statistics in the prior art, figure number statistics is inaccurate, reference resource allocation is affected, limitation exists when model application is adopted for figure recognition, and application in an actual scene is limited. The method comprises the following steps: periodically scanning a time node to collect and monitor a first video image; performing character recognition and counting on the first video image by adopting a character recognition model fusing self-attention and convolutional deep learning; analyzing the character motion trend to dynamically count the number of people in the monitoring area; and calculating the number of users in the monitoring area, and activating corresponding base station capacity resources according to the number of users. According to the invention, the resources of the base station are reasonably allocated by using long-term monitoring equipment, identifying the character condition in the image and combining the related settings of the base station, so that the energy consumption of the 5G base station is further reduced, the overall operation cost is reduced, and the green and energy-saving effects are achieved.

Description

Technical Field

[0001] The present invention relates to the field of communication technology, and in particular to a green base station energy-saving method based on the fusion of self-attention and convolutional deep learning. Background Art

[0002] With the development of mobile communication technology, the number of base stations is increasing, and its energy consumption problem is becoming increasingly prominent. At present, the energy consumption of 5G base station equipment accounts for a considerable proportion of the energy consumption of the entire wireless network. In order to achieve green energy saving of base stations, it is necessary to find an effective method to check the number of users and adjust the working mode of the base station accordingly.

[0003] In the prior art, there is a method of identifying people and counting the number of people through a person recognition model, and allocating 5G base station resources according to the number of people to achieve the purpose of energy saving. The existing person recognition models are already able to realize the ability to identify and locate a variety of different targets, but the application of the computing methods used by these models has certain limitations, which restricts their application in actual scenarios. For example, the feature pyramid structure is used for feature fusion, but various feature pyramid network structures cannot effectively utilize the correlation between all pyramid feature maps. Another example is the inherent limitations of convolution, including the independence of the local receptive field and the input content, which limits the model's ability to capture long distances. Summary of the invention

[0004] The present invention mainly solves the problems that the prior art allocates base station resources by counting the number of people, the counting of the number of people is inaccurate, which affects the allocation of benchmark resources, and the application of models for character recognition has limitations, which restricts its application in actual scenarios. A green base station energy-saving method based on the fusion of self-attention and convolutional deep learning is provided.

[0005] The above technical problem of the present invention is mainly solved by the following technical solution: a green base station energy saving method based on the fusion of self-attention and convolutional deep learning, comprising the following steps: S1. Periodic scanning time node acquisition monitoring first video image; S2. Using a person recognition model that combines self-attention and convolutional deep learning to perform person recognition and counting on the first video image; S3. Analyze the movement trend of people and dynamically count the number of people in the monitoring area; S4. Calculate the number of users in the monitoring area and activate the corresponding base station capacity resources according to the number of users.

[0006] The present invention uses long-term monitoring equipment to identify the situation of people in the image, and combines the relevant settings of the base station to reasonably allocate the resources of the base station, so that the 5G base station can further reduce energy consumption and reduce the overall operating cost, thereby achieving green energy saving. Through the analysis of the movement trend of the person, the person's appearance in the monitoring image is predicted, and the number of users in the monitoring area is dynamically counted according to the actual movement of the person, which improves the accuracy of the person number statistics, so that the base station resources can be more accurately allocated according to the number of people, the energy utilization efficiency of the base station can be improved, the energy consumption of the base station can be reduced, and the overall operating cost can be reduced.

[0007] As a preferred solution, the person recognition model integrating self-attention and convolutional deep learning includes: The input image is converted into feature maps of multiple scale spaces through the backbone network, and the first features of each scale space are extracted and output respectively; Through the feature fusion neck, shallow feature interaction and deep feature interaction are performed on the first features of each scale space, and the first features of each scale space are spliced. According to the deep interaction and splicing features, context fusion is performed to output the second features of each scale space respectively; Each second feature input prediction head generates a multi-target output.

[0008] The backbone network converts the input image into multiple first features of different scales with low spatial resolution but rich semantic information. Each first feature is input into the feature fusion neck respectively, and each first feature is input into a shallow feature interaction module for shallow feature interaction. Shallow feature interaction is responsible for cross-scale information exchange and fusion. The main goal is to integrate the shallow features of the previous layer with the high-resolution shallow features from the same layer and the backbone network, aiming to retain rich positioning details to enhance the spatial performance of the network. After the shallow feature interaction module, it enters the cross-stage block module. The cross-stage block module is a multi-branch structure that allows interactions between features at different levels. After the cross-stage block module, it enters the deep feature interaction module for deep feature interaction, using additional deep information to enhance feature representation, mainly for shallow high-resolution layers, shallow low-resolution layers, shallow layers of the same level, and the previous layer. The gradient information of the output layer is enriched through multi-directional connections. In addition, the first features of each scale space are spliced ​​through a ternary scale sequence fusion module. Specifically, the feature maps of different scales are stacked horizontally, and their scale sequence features are extracted using three-dimensional convolution.

[0009] As a preferred solution, the backbone network includes three high-efficiency layer aggregation modules connected in sequence, which extract features and output first features of different scales. The bottom-level high-efficiency layer aggregation module obtains feature maps through local-global hybrid self-attention mechanism and relative position enhanced self-attention mechanism, and stacks the feature maps of different scales on the feature map to output the first feature.

[0010] The backbone network converts the input image into multiple feature maps of different scales with low spatial resolution but rich semantic information. The output feature maps are fed into three efficient layer aggregation modules K1, K2, and K3, which use attention mechanisms or other methods to selectively focus on important features.

[0011] As a preferred solution, the efficient layer aggregation module performs point-by-point convolution and feature separation operations on the input information, and after the operation, it is divided into a first branch and a second branch. The first branch enters the feature stack after N inverted residual bottlenecks, and the second branch retains the original information and enters the feature stack. Finally, point-by-point convolution is performed to output the information. The local-global hybrid self-attention mechanism includes three input branches. The first input branch outputs global information after passing through a linear projection layer, layer normalization, one-dimensional convolution, layer normalization, and compressed excitation attention. The second input branch outputs local information after passing through a linear projection layer, layer normalization, multi-head self-attention, layer normalization, and compressed excitation attention. The global information, local information, and original input are input into an adaptive hybrid global and local information module to form a hybrid output. The hybrid output is transformed by a linear projection layer and then added to the original input to form a residual connection. The output information is processed by layer normalization. The relative position enhanced self-attention mechanism inputs enter the query matrix linear projection layer, the key matrix linear projection layer, the value matrix linear projection layer, and the parameter matrix linear projection layer respectively. The outputs of the matrix linear projection layer and the key matrix linear projection layer are matrix multiplied and added to the relative position of the bias. After being processed by the softmax function, they are matrix multiplied with the output of the value matrix linear projection layer to form a first output. The output of the parameter matrix linear projection layer acts on the tokens depth-separable convolution, and is linearly projected at the tokens level to form a second output. The first output is added to the second output to obtain the final output.

[0012] The efficient layer aggregation module in this scheme can efficiently learn expressive multi-scale feature representations. The input information is divided into two branches after point-by-point convolution and feature separation operations. One branch retains the original information and then directly enters the feature stacking operation, while the other branch is processed by N inverted residual bottleneck units. Due to the mechanism of efficient layer aggregation, the branches and the outputs through each inverted residual bottleneck unit are retained and finally connected together. The specific structure of the inverted residual bottleneck unit, the input sequence is expanded by point-by-point convolution to expand the number of channels, followed by k*k heterogeneous depthwise separable convolution operations, and finally point-by-point convolution is used to reduce the number of channels and compensate for the information loss caused by the depthwise separable convolution. During the training of the inverted residual bottleneck, the network runs n depthwise convolutions of different sizes in parallel. This module expands the perception range and retains the information of small objects without incurring additional inference costs by parallelizing large kernel convolutions with several small kernel convolutions.

[0013] The local-global mixed self-attention mechanism jointly models the user's long-term and short-term preferences by combining local convolutional layers and global self-attention layers. The local preferences (captured by CNN) and global preferences (captured by Transformer) are combined to more comprehensively model the user's dynamic preferences; it adaptively mixes global and local information, decouples the fusion process in different layers, improves the expressiveness, and adaptively aggregates long-term and short-term preferences (the mixed importance of local and global dependency modules can be adjusted according to the user's personalized needs); compressed incentive attention is used to replace the softmax operation, allowing multiple related items to be considered at the same time, enhancing the model's expressiveness (allowing the model to focus on multiple highly related objects at the same time, instead of focusing on only a single object like traditional softmax).

[0014] Compressed Incentive Attention: Takes an input tensor of shape [B, N, D], where B is the batch size, N is the sequence length, and D is the feature dimension.

[0015] First, the average value of the input tensor along the feature dimension D is calculated to obtain the average value of the tensor with shape [B, N, 1].

[0016] Transpose the dimensions of the input tensor mean to obtain a tensor of shape [B, 1, N].

[0017] The transposed tensor is transformed through the first linear layer to obtain a tensor hidden layer state with a shape of [B, 1, N / r], where N / r represents the compressed dimension.

[0018] Apply an activation function (ReLU) to further nonlinearly activate the hidden layer states.

[0019] The activated tensor is transformed back to the original sequence length [B, 1, N] through a second linear layer.

[0020] Apply the Sigmoid function to generate the attention weight tensor attention score, with a shape of [B, 1, N].

[0021] The attention scores are transposed again so that they can be multiplied with the original input tensor, resulting in a tensor of shape [B, N, 1].

[0022] Adaptively mix global and local information, receiving three input tensors: model input, global information, and local information.

[0023] Calculates the average of the input tensor along the feature dimension for generating adaptive weights.

[0024] The adaptive weight alpha is obtained through a linear projection layer and compressed excitation attention and expanded to a [B, 1,1] shape (where B represents the input sample batch size).

[0025] beta is set to 1-alpha, representing the remaining proportion.

[0026] According to these two weights, the global information and the local information are mixed together to form a mixed output.

[0027] The mixed output is transformed by the linear projection layer and then added to the original input to form a residual connection.

[0028] The final output is processed through a dropout layer and layer normalization.

[0029] Dynamically adjust the weight between global and local information. Use residual connections to keep gradients flowing and reduce the risk of gradient vanishing. Enhance the generalization ability of the model through layer normalization and dropout.

[0030] Relative Position Enhanced Self-Attention Mechanism, Importance of Position Information, For computer vision tasks, pixels or patches in an image are regarded as tokens, and their position information is crucial for constructing image semantics. A key feature of the self-attention mechanism is that it is insensitive to the order of the tokens, which means that it cannot directly utilize the position information of the tokens. Introducing relative position bias: Methods such as Swin Transformer guide the model to learn position information and content-based attention by introducing an additional relative position bias in the attention matrix. The relative position enhanced self-attention mechanism further develops this idea by establishing an independent value representation space to capture position dependencies. Mathematical formula of the relative position enhanced self-attention mechanism: The relative position enhanced self-attention mechanism transforms the token into a new representation by introducing a parameter matrix Vrel in order to mix information by relative position. The relative position enhanced self-attention mechanism uses a weight tensor brel, which is indexed by relative position. The relative position enhanced self-attention mechanism also retains the relative position bias B to give the model greater flexibility. Limiting the maximum absolute distance: In order to enhance the generalization ability of the model and simplify the implementation, the relative position enhanced self-attention mechanism limits the maximum absolute relative position distance s on each axis. After doing so, the computation of the relative position enhanced self-attention mechanism can be simplified to a token-based convolution with kernel size (2s+1)*(2s+1).

[0031] As a preferred solution, the feature fusion neck includes three layers of branches, each layer of branches includes a shallow feature interaction module, a cross-stage blocking module, a deep feature interaction module, and a spatial context perception module in sequence. The shallow feature interaction module receives the first features of the same layer and the previous layer, and outputs the shallow features to the cross-stage blocking module. The cross-stage blocking module distributes the shallow features to the front deep feature interaction modules and the rear upper shallow feature interaction module. The deep feature interaction module outputs the deep features to the spatial context perception module. The ternary scale sequence fusion module obtains the first features of the same layer of each branch, and after feature splicing, it is input into the first layer of spatial context perception module. The spatial context perception module outputs the global context features to the rear next layer of deep feature interaction module and to the prediction head.

[0032] After each efficient layer aggregation module, there is a shallow feature interaction module responsible for cross-scale information exchange and fusion. The shallow feature interaction module is followed by the cross-stage block module, which is a multi-branch structure that allows interaction between features at different levels. The deep feature interaction module utilizes additional deep information to enhance feature representation. The spatial context-aware module may capture spatial relationships and generate richer feature representations.

[0033] As a preferred solution, the shallow feature interaction module receives the first features of the previous layer and the same layer, and the shallow features of the next layer. The first features of the previous layer are subjected to depth-separable convolution to form deep information, which is stacked with the first features of the same layer and the shallow features of the next layer to output shallow features; The deep feature interaction module receives the shallow features of the previous layer, the same layer, and the next layer, and the global context features of the previous layer, the shallow features of the previous layer, the shallow features of the next layer, and the global context features of the previous layer, which are respectively subjected to depth-separable convolution and then stacked with the shallow features of the same layer to output deep features; The ternary scale sequence fusion module includes three layers of branches, which respectively receive three layers of first features of different scales, each of which passes through a convolution module to adjust the channel. The first layer of branches is downsampled by maximum pooling and average pooling, and then processed by the convolution module. The third layer of branches is upsampled by the nearest neighbor interpolation method, and then processed by the convolution module. The three layers of branch features are then stacked and processed by the three-dimensional convolution module to output the spliced ​​features. The context-aware module includes three branches, which respectively perform global average pooling, global maximum pooling, and point-by-point convolution on the input features, and then perform softmax function, matrix multiplication, and point-by-point convolution on the global average pooling and global maximum pooling. The point-by-point convolution is processed by softmax function, matrix multiplication, and point-by-point convolution, and finally perform Hadamard product together to output global context features.

[0034] Shallow feature interaction module, accurate positioning depends on detailed edge information obtained from the shallow network, while accurate classification requires a deeper network to capture coarse-grained information. An effective feature pyramid network should support the full and sufficient fusion of shallow and deep network information flows; retaining shallow spatial information in the backbone network is crucial to enhancing the detection ability of smaller targets. However, the information provided by the backbone network is relatively basic and easily disturbed. Therefore, the shallow information is integrated into the deep network as an auxiliary branch to ensure the stability of subsequent layer learning; the main goal is to integrate deep information with high-resolution shallow features from the same layer and the next layer in the backbone network, aiming to retain rich positioning details to enhance the spatial performance of the network; use 1*1 convolution to control the number of channels in the shallow information to ensure that it accounts for a small proportion in the splicing operation and does not affect subsequent learning. The shallow backbone information is maintained through bidirectional connections, which enhances the network's ability to detect small targets.

[0035] The deep feature interaction module performs multi-scale information fusion at a deeper level to further improve the interactive use of feature layer information. These connections span the shallow high-resolution layer, the shallow low-resolution layer, the shallow layer of the same level, and the front spatial context perception to aggregate information. The gradient information of the output layer is enriched through multi-directional connections.

[0036] Ternary Scale Sequence Fusion Module,In order to identify densely overlapping small objects, the shape or appearance changes at different scales can be referenced and compared by enlarging the image. Since different feature layers of the backbone network have different sizes, the traditional feature pyramid fusion mechanism only upsamples the feature maps of small sizes and then splits or adds them to the features of the previous layer, ignoring the rich detail information of the large-size feature layer. The ternary scale sequence fusion module can split the features of large, medium, and small sizes, add the feature maps of large sizes, and enlarge the features to improve the detailed feature information. It can better combine the high-dimensional information of the deep feature map and the detailed information of the shallow feature map. The scale space is constructed along the scale axis of the image, which not only represents a scale, but also represents the range of various scales that an object may have. Feature maps of different scales are stacked horizontally, and their scale sequence features are extracted using 3D convolution. The high-resolution feature map contains most of the information that is crucial for the detection of small objects. Before feature encoding, the number of feature channels should be adjusted to keep it consistent with the main scale features. After the convolution module processes the large-size feature map, its number of channels is adjusted to 1C, and then a hybrid structure of maximum pooling and average pooling is used for downsampling, which helps to retain the effectiveness and diversity of high-resolution features and cell images. For small-size feature maps, the convolution module is also used to adjust the number of channels, and then the nearest neighbor interpolation method is used for upsampling. This helps to maintain the richness of local features of low-resolution images and prevent the loss of feature information of small targets. Finally, the three feature maps of the same size, large, medium, and small, are convolved once, then spliced ​​in the channel dimension, and finally three-dimensional convolution, batch normalization, and activation function are performed.

[0037] The spatial context-aware module is designed to model global context information across channels and spatial dimensions and suppress useless background features. The structure consists of three branches, using global average pooling (GAP) and global maximum pooling (GMP) to integrate global information. The functions are: extracting contextual relationships between space and channels. Enhance the feature representation of small objects and reduce background confusion. Implement linear transformations that simplify feature maps using 1x1 convolutions. Calculate channel and spatial context information through matrix multiplication. Hadamard product operation, output feature maps containing global context.

[0038] As a preferred solution, step S3 specifically includes: S31. Analyze the motion state of the person through deep learning, determine the moving direction of the person in motion, calculate the moving trajectory of the person, and determine whether the person is about to exceed the current video image range; S32. If not, count the number of people in each monitoring area and proceed to step S4; If yes, determine the range where the person is about to appear based on the task movement trajectory, obtain the second video image of the monitoring after the set delay time, perform person recognition and counting on the second video image, count the number of people in each monitoring area, and enter step S4, while judging whether the current time exceeds the current scanning cycle; S33. If yes, enter the next scanning cycle and return to step S1; If not, return to step S31.

[0039] This method arranges cameras in a set monitoring area, using one or more cameras, and the cameras may cover the monitoring area. For example, a camera is set, and the coverage of the camera is set as the monitoring area, or multiple cameras are set, and the coverage area of ​​multiple cameras constitutes a monitoring area. The camera sets a scanning cycle, and regularly scans or detects images to count the number of users. When it is analyzed that the person is in motion, that is, the person may walk out of the current monitoring area and enter other monitoring areas, and different monitoring areas may be in the coverage range of different base stations. Therefore, it is necessary to re-count the number of people in the monitoring area according to the dynamic movement of the person, so that the base station resources can be more accurately allocated according to the number of people. In this scheme, the movement trend of the person is analyzed through deep learning, and the number of people in the monitoring area is dynamically counted. The movement trend of the person in the current video image is analyzed through deep learning, and the movement trend of the person is used to determine whether it is about to exceed the current video image range. In addition, it further includes analyzing non-full-body people in the video image, and non-full-body people are also considered to be about to exceed the video image range. The movement trend of the person is determined by analyzing the movement state of the person, determining the moving direction of the person in the moving state, and calculating the moving trajectory of the person, so as to judge whether it will exceed the range of the video image. At the same time, according to the moving direction and moving trajectory, it can also predict the camera coverage range where the person will appear in the surrounding area, so that the video image of the surrounding camera can be retrieved to determine whether the person has moved over. If it is judged that there is a person in the video image that will exceed the range of the video image, the second video image of the monitoring range where the task will appear is obtained after the set delay time. The delay time is set according to the demand, and the delay time is less than the scanning cycle length. For example, the delay time can be set according to the frequency of the camera capturing the video frame, such as the camera captures the video frame once every minute, and the delay time is set to one minute. After obtaining the second video image, the person is identified and counted through the person recognition model, and the position of each person is marked. At the same time, the second video image is also subjected to deep learning analysis to identify the person who moves to the current image range. At this time, the number of users in each monitoring area is counted again by the number of people calculated by the monitoring camera, and the base station resources are dynamically adjusted according to the number of users. At the same time, the movement trend of the person in the second video image is analyzed, and then it is determined whether there is a person about to exceed the range of the second video image. If the person who moves to the current image range keeps moving, or other people have a tendency to move, the above operation is repeated, and the number of people in the monitoring area is dynamically counted according to the movement trend of the person, and the base station resources are adjusted in real time until the person in the video image of the surrounding monitoring obtained after the delay time no longer moves or the current time exceeds a scanning cycle, then the operation is stopped and the next scanning cycle is entered, and the operation of the next scanning cycle is performed. If multiple people are detected to be about to exceed the image range in the first video image, and other people are detected to be about to exceed the image range in the second video image, these people are tracked respectively, and step S3 is repeated.This solution uses a small monitoring area, which can be composed of one or a few monitoring cameras. The movement of people will not be too complicated, and the base station resources can be adjusted in time according to the movement of people to achieve the purpose of reasonable resource allocation.

[0040] As a preferred solution, after step S2, the following steps are further included: Determine whether there is a person in the character recognition result. If yes, go to step S3; if no, notify the 5G base station to enter the waiting, sleep or deep sleep state.

[0041] After character recognition, this solution first determines whether a person appears. When no task appears, the 5G base station is notified to enter a waiting, sleep or deep sleep state. When a person is detected, the coverage base station is activated, effectively improving the energy utilization efficiency of the base station, reducing the energy consumption of the base station, and reducing the overall operating cost.

[0042] As a preferred solution, the calculation of the number of users in the monitoring area specifically includes: Obtain the operator user penetration rate in this business area and calculate the number of operator users in each monitoring area; Get the operator's 5G user ratio and calculate the operator's 5G users in each monitoring area.

[0043] In order to count the number of 5G users, it is necessary to input the operator data of the business area as the basis for calculation and analysis. The number of operator users can be calculated by inputting the operator user penetration rate in the business area, and the number of 5G users can be calculated by inputting the proportion of operator 5G users.

[0044] As a preferred solution, a real-time adjustment step is also included, specifically including: Calculate the expected capacity requirements of users based on the number of users in the monitoring area, and obtain the actual capacity requirements of users in the monitoring area; Compare the user's expected capacity demand with the user's actual capacity demand, and adjust the base station capacity resources in real time according to the user's actual capacity demand.

[0045] This solution is performed after step S5. Since users have the problem of using large or small capacity in actual situations, it is necessary to comprehensively count and analyze the capacity requirements of users in order to better plan and adjust resources. The expected capacity requirements of users are estimated based on the number of users counted in the monitoring area, and the actual capacity requirements of users in the monitoring range are obtained through the base station. The actual capacity requirements of users are calculated based on the positions of people marked by the character recognition model, and then the users in the monitoring area are obtained through the positions, and the actual capacity requirements of these users are counted from the base station. The actual capacity requirements of users are compared with the expected capacity requirements of users. If the actual capacity requirements of users are greater than the expected capacity requirements of users, the base station capacity resources are increased according to the actual capacity requirements of users. If the actual capacity requirements of users exceed the current base station resource capacity, the base station resources are considered to be increased to meet the requirements. If the actual capacity requirements of users are less than the expected capacity requirements of users, no operation is performed. In addition, in order to further improve energy utilization efficiency and reduce base station energy consumption, if the actual capacity requirements of users are less than the expected capacity requirements of users, the base station capacity requirements can also be reduced according to the actual capacity requirements of users. After the real-time adjustment step is completed, a round of adjustment of base station resources according to the number of people is completed.

[0046] Therefore, the advantages of the present invention are: By using long-term monitoring equipment to identify people in images and combining relevant settings of base stations, the resources of base stations can be reasonably allocated, so that 5G base stations can further reduce energy consumption and reduce overall operating costs, thus achieving green energy conservation. Through the analysis of people's movement trends, the appearance of people in the monitoring images is predicted, and the number of users in the monitoring area is dynamically counted according to the actual movement of the people, which improves the accuracy of the number of people. Therefore, the base station resources can be more accurately allocated according to the number of people, the energy utilization efficiency of the base station can be improved, the energy consumption of the base station can be reduced, and the overall operating cost can be reduced.

[0047] The person recognition model adopts a ternary scale sequence fusion model to segment large, medium and small features, add large-sized feature maps, and amplify features to improve detailed feature information, and can better combine the high-dimensional information of deep feature maps and the detailed information of shallow feature maps; it adopts a relative position enhanced self-attention mechanism to make up for the position blind spot of self-attention, and the model has a strong ability to capture dependencies from content and position; it adopts a spatial context perception module to extract the contextual relationship between space and channels, enhance the feature representation of small targets, and reduce background confusion; it adopts a local-global mixed self-attention mechanism to combine local preferences with global preferences to more comprehensively model users' dynamic preferences. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a schematic flow chart of the method of the present invention.

[0049] Figure 2 It is an architecture diagram of the person recognition model in the present invention.

[0050] Figure 3 It is a structural diagram of the efficient layer aggregation module of the person recognition model in the present invention.

[0051] Figure 4 It is a structural diagram of the local-global hybrid self-attention mechanism of the person recognition model in the present invention.

[0052] Figure 5 It is a structural diagram of the relative position enhanced self-attention mechanism of the person recognition model in the present invention.

[0053] Figure 6 It is a structural diagram of the shallow feature interaction module of the person recognition model in the present invention.

[0054] Figure 7 It is a result diagram of the deep feature interaction module of the person recognition model in the present invention.

[0055] Figure 8 It is a structural diagram of the ternary scale sequence fusion module of the person recognition model in the present invention.

[0056] Fig. 9 It is a structural diagram of the spatial context perception module of the person recognition model in the present invention. DETAILED DESCRIPTION

[0057] The technical solution of the present invention is further specifically described below through embodiments and in conjunction with the accompanying drawings.

[0058] Example: This embodiment provides a green base station energy saving method based on the fusion of self-attention and convolutional deep learning, such as Figure 1 As shown, the following steps are included: S1. Periodic scanning time node acquisition monitoring first video image; S2. Using a person recognition model that combines self-attention and convolutional deep learning to perform person recognition and counting on the first video image; S3. Analyze the movement trend of people and dynamically count the number of people in the monitoring area; S4. Calculate the number of users in the monitoring area and activate the corresponding base station capacity resources according to the number of users.

[0059] As a preferred solution of this embodiment, step S1 specifically includes setting a scanning cycle schedule, acquiring the first video image of the surveillance camera at the time node at the beginning of each scanning cycle, and performing subsequent monitoring and analysis. As a basis for the implementation of the method, a surveillance area is delineated, and surveillance cameras are installed at key locations in each surveillance area, hereinafter referred to as surveillance. One or more surveillance cameras are installed in one surveillance area, and one or more surveillance cameras cover the surveillance area as much as possible. The surveillance is set to capture video frames at a fixed frequency, such as once per minute.

[0060] As a preferred solution of this embodiment, in step S2, the first video image is subjected to character recognition by adopting a character recognition model that integrates self-attention and convolutional deep learning, marking pedestrians of each character, and calculating the number of characters in the first video image. The character recognition model is a character recognition model for detecting the number of people based on the integration of self-attention and convolutional deep learning.

[0061] As a preferred solution of this embodiment, firstly, it is determined whether a person appears in the first video image according to the person recognition result; If yes, then go to step S3 to conduct further character behavior pattern analysis; If not, notify the 5G base station to enter the waiting, sleep or deep sleep state.

[0062] In this solution, the operation of the base station is controlled by whether there is someone there. When no one is around, the base station is put into waiting or sleep mode. When a person is detected, the base station is activated, which effectively improves the energy efficiency of the base station, reduces the energy consumption of the base station, and reduces the overall operating cost.

[0063] As a preferred solution of this embodiment, step S3 specifically includes: S31. Analyze the motion state of the person through deep learning, determine the moving direction of the person in motion, calculate the moving trajectory of the person, and determine whether the person is about to exceed the current video image range; S32. If not, count the number of people in each monitoring area and proceed to step S4; If yes, determine the range where the person is about to appear based on the task movement trajectory, obtain the second video image of the monitoring after the set delay time, perform person recognition and counting on the second video image, count the number of people in each monitoring area, and enter step S4, while judging whether the current time exceeds the current scanning cycle; S33. If yes, enter the next scanning cycle and return to step S1; If not, return to step S31.

[0064] Further analysis of the current video image shows that the person is in motion, that is, the person may walk out of the current monitoring area and enter other monitoring areas. Different monitoring areas may be in different base station coverage areas, which will change the user capacity demand within the baseline coverage area. Therefore, it is necessary to re-count the number of people in the monitoring area based on the movement of the person, so as to more accurately allocate base station resources based on the number of people. In this solution, deep learning is used to analyze the movement trend of people and dynamically count the number of people in the monitoring area. Deep learning analysis uses the following technologies: Object Tracking Algorithms: DeepSORT: DeepSORT is an algorithm that combines deep learning feature extraction and traditional SORT (Simple Online and Realtime Tracking) tracker. It can be used to track the position and motion trajectory of multiple targets in a video sequence.

[0065] FairMOT: FairMOT is an algorithm for multi-target tracking that can detect and track simultaneously in a single inference, suitable for real-time application scenarios.

[0066] Action Recognition Algorithms: Spatial-temporal Graph Convolutional Networks (ST-GCN): This type of algorithm analyzes human motion by constructing spatiotemporal graphs and is suitable for understanding complex behavioral patterns.

[0067] Two-stream Convolutional Networks: This algorithm combines RGB image stream and optical flow to capture spatial and temporal information in the video, which is very effective for understanding the movement and direction of the character.

[0068] Person Re-Identification (ReID) algorithm: Deep Person Re-ID: This type of algorithm can recognize the same individual by learning identity-invariant features across camera views, which is helpful for tracking the movement path of people between different cameras.

[0069] Trajectory Prediction Algorithms: Social LSTM: This algorithm takes into account the social interactions between pedestrians and can predict their future trajectories.

[0070] ConvLSTM: Convolutional LSTM combines the advantages of convolutional neural network (CNN) and long short-term memory network (LSTM) and can be used to predict the movement trajectory of a person.

[0071] Through deep learning, the movement trend of the person in the current video image is analyzed, and the movement trend of the person is used to determine whether the person is about to exceed the current video image range. Further judgment of whether the person is about to exceed the current video image range also includes analyzing the non-full-body person in the video image, and the non-full-body person is also considered to be about to exceed the video image range. The movement trend of the person is determined by analyzing the movement state of the person, determining the moving direction of the person in the moving state, and calculating the moving trajectory of the person, so as to determine whether the person will exceed the video image range. At the same time, according to the moving direction and moving trajectory, it can also be predicted that the person will appear in the surrounding camera coverage range, so that the video image of the surrounding camera can be retrieved to determine whether the person has moved over. When it is determined that there is a person in the video image that will exceed the video image range, the second video image of the monitoring range where the task will appear is obtained after the set delay time. The delay time is set according to the demand, and the delay time is less than the scanning cycle length. For example, the delay time can be set according to the frequency of the camera capturing video frames. For example, the camera captures a video frame every one minute, and the delay time is set to one minute. After obtaining the second video image, the characters are identified and counted through the character recognition model, and the position of each character is marked. At the same time, the second video image is also subjected to deep learning analysis to identify the characters that move to the current image range. At this time, the number of people calculated by the monitoring camera is re-counted to count the number of users in each monitoring area, and the base station resources are dynamically adjusted according to the number of users. At the same time, the movement trend of the characters in the second video image is analyzed, and then it is determined whether there are characters that are about to exceed the range of the second video image. For example, the characters that move to the current image range keep moving, or other characters have a tendency to move. The above operation is repeated, and the number of people in the monitoring area is dynamically counted according to the movement trend of the characters, and the base station resources are adjusted in real time until the characters in the video images of the surrounding monitoring obtained after the delay time no longer move or the current time exceeds a scanning cycle, then the operation is stopped and the next scanning cycle is entered, and the operation of the next scanning cycle is performed. In this scheme, a small monitoring area can be set, which can be composed of one or a few monitoring cameras. The movement of characters will not be too complicated, and the base station resources can be adjusted in time according to the movement of characters to achieve the purpose of reasonable resource allocation.

[0072] As a preferred solution of this embodiment, step S4 specifically includes: The number of users is calculated based on the number of people in each monitored area. Specifically, the operator user penetration rate in this business area is obtained, and the number of operator users in each monitored area is calculated by multiplying it by the number of people in the monitored area. The proportion of 5G users of the operator is obtained and multiplied by the number of operator users to calculate the number of operator 5G users in each monitored area.

[0073] Prepare the corresponding benchmarks in advance, enter the number of base stations covering the site, understand the base station coverage of each monitoring area, and prepare for subsequent resource allocation. Further refine the base station coverage for more accurate analysis. According to the number of 5G users of the operator obtained, activate the corresponding base station capacity resources to ensure smooth network operation.

[0074] Furthermore, it is necessary to adjust base station resources in real time according to actual user conditions. Since users may use too much or too little capacity in actual situations, it is necessary to comprehensively count and analyze users' capacity requirements in order to better plan and adjust resources. The real-time adjustment steps include: Calculate the expected capacity requirements of users based on the number of users in the monitoring area, and obtain the actual capacity requirements of users in the monitoring area; Compare the user's expected capacity demand with the user's actual capacity demand, and adjust the base station capacity resources in real time according to the user's actual capacity demand.

[0075] Specifically, the expected capacity demand of users is estimated based on the number of users counted in the monitoring area, and the actual capacity demand of users within the monitoring range is obtained through the base station. The actual capacity demand of users is calculated based on the position of the person marked by the person recognition model, and then the users in the monitoring area are obtained through the position, and the actual capacity demand of these users is counted from the base station. The actual capacity demand of users is compared with the expected capacity demand of users. If the actual capacity demand of users is greater than the expected capacity demand of users, the base station capacity resources are increased according to the actual capacity demand of users. If the actual capacity demand of users exceeds the current base station resource capacity, consider increasing the base station resources to meet the demand. If the actual capacity demand of users is less than the expected capacity demand of users, no operation is performed. In addition, in order to further improve energy utilization efficiency and reduce base station energy consumption, if the actual capacity demand of users is less than the expected capacity demand of users, the base station capacity demand can also be reduced according to the actual capacity demand of users. After the real-time adjustment step is completed, a round of adjustment of base station resources according to the number of people is completed.

[0076] The present invention uses long-term monitoring equipment to identify the situation of people in the image, and combines the relevant settings of the base station to reasonably allocate the resources of the base station, so that the 5G base station can further reduce energy consumption and reduce the overall operating cost, thereby achieving green energy saving. Through the analysis of the movement trend of the person, the person's appearance in the monitoring image is predicted, and the number of users in the monitoring area is dynamically counted according to the actual movement of the person, which improves the accuracy of the person number statistics, so that the base station resources can be more accurately allocated according to the number of people, the energy utilization efficiency of the base station can be improved, the energy consumption of the base station can be reduced, and the overall operating cost can be reduced.

[0077] The following scenario example is used to illustrate the method steps of this embodiment, such as a 5G base station energy-saving system near a shopping mall (concert venue, etc.). It is assumed that a green base station energy-saving system based on the fusion of self-attention and convolutional deep learning is deployed near a large shopping mall. This shopping mall has few people on weekdays, but a surge in traffic on weekends and holidays.

[0078] Initial setup and monitoring Install high-definition surveillance cameras at key locations around the shopping mall to cover as many pedestrian passages as possible. These surveillance cameras capture video frames at a fixed frequency, such as once a minute. Set the scanning cycle, such as automatically scanning every 10 minutes with the preset, to obtain the latest pedestrian flow data.

[0079] By using a pre-trained person recognition model to identify people in the images captured by the surveillance, the model distinguishes pedestrians from background objects, marks the position of each pedestrian, and calculates the number of pedestrians in the image. It determines whether there are people. If there are people in the image, it proceeds to the next step of analysis. If there are no pedestrians, the 5G base station is notified to enter energy-saving mode such as sleep mode.

[0080] Deep learning analysis can further analyze pedestrian behavior patterns, such as direction of travel, speed, and degree of aggregation, which can help predict pedestrian flow trends in the future.

[0081] Based on the analysis results, the headcount is corrected and refined, and the number of people in each monitoring area is counted.

[0082] Output the headcount data and send the final headcount data to the control center of the system as the basis for adjusting the working status of the base station.

[0083] Business area and operator data input Enter operator user penetration: Collect wireless signal strength data and other network performance indicators within the shopping mall area to calculate the number of operator users.

[0084] Input 5G user share ratio: Understanding the proportion of 5G users of different operators in the area and calculating the number of 5G users of operators will help to allocate network resources more reasonably.

[0085] Enter the number of base stations covering the area: Confirm all base stations covering the area and understand their respective coverage areas.

[0086] Resource Allocation and Notification Based on the calculated total number of 5G users in the current area, the corresponding base station capacity resources are activated, including: dynamically adjusting the workload of the base station according to changes in the number of users, such as increasing bandwidth or adjusting signal transmission power.

[0087] Statistics on user capacity requirements: Regularly update user capacity requirements and optimize base station configuration accordingly.

[0088] Increase base station resources: If it is found that user demand exceeds existing resources, the system will notify the administrator to add new base stations or enhance the functions of existing base stations.

[0089] Notify the 5G base station management platform: Ensure that all adjustments are synchronized to the 5G base station management platform so that it can respond to changes in user needs in a timely manner.

[0090] Through such a system, not only can unnecessary energy consumption be effectively reduced, but also the user experience can be improved and sufficient network capacity support can be ensured during peak hours.

[0091] In step S2, a person recognition model based on the fusion of self-attention and convolutional deep learning is used for person detection, such as Figure 2 As shown, the person recognition model includes: The input image is converted into feature maps of multiple scale spaces through the backbone network, and the first features of each scale space are extracted and output respectively; Through the feature fusion neck, shallow feature interaction and deep feature interaction are performed on the first features of each scale space, and the first features of each scale space are spliced. According to the deep interaction and splicing features, context fusion is performed to output the second features of each scale space respectively; Each second feature input prediction head generates a multi-target output.

[0092] The backbone network converts the input image into multiple first features of different scales with low spatial resolution but rich semantic information. Each first feature is input into the feature fusion neck respectively, and each first feature is input into a shallow feature interaction module for shallow feature interaction. Shallow feature interaction is responsible for cross-scale information exchange and fusion. The main goal is to integrate the shallow features of the previous layer with the high-resolution shallow features from the same layer and the backbone network, aiming to retain rich positioning details to enhance the spatial performance of the network. After the shallow feature interaction module, it enters the cross-stage block module. The cross-stage block module is a multi-branch structure that allows interactions between features at different levels. After the cross-stage block module, it enters the deep feature interaction module for deep feature interaction, using additional deep information to enhance feature representation, mainly for shallow high-resolution layers, shallow low-resolution layers, shallow layers of the same level, and the previous layer. The gradient information of the output layer is enriched through multi-directional connections. In addition, the first features of each scale space are spliced ​​through a ternary scale sequence fusion module. Specifically, the feature maps of different scales are stacked horizontally, and their scale sequence features are extracted using three-dimensional convolution.

[0093] Specifically, the backbone network includes three efficient layer aggregation modules connected in sequence. The efficient layer aggregation modules extract features and output first features of different scales. The lowest efficient layer aggregation module obtains feature maps through local-global hybrid self-attention mechanism and relative position enhanced self-attention mechanism, and stacks them with feature maps of different scales on the upper layer of the feature map to output the first feature.

[0094] like Figure 3 As shown in the figure, the efficient layer aggregation module performs point-by-point convolution and feature separation operations on the input information. After the operation, it is divided into a first branch and a second branch. The first branch enters the feature stack after N inverted residual bottlenecks, and the second branch retains the original information and enters the feature stack. Finally, point-by-point convolution is performed and the information is output.

[0095] The efficient layer aggregation module can efficiently learn expressive multi-scale feature representations. The input information is divided into two branches after point-by-point convolution and feature separation operations. One branch retains the original information and then directly enters the feature stacking operation, while the other branch is processed by N inverted residual bottleneck units. Due to the mechanism of efficient layer aggregation, the branches and the outputs through each inverted residual bottleneck unit are retained and finally connected together. The specific structure of the inverted residual bottleneck unit, the input sequence is expanded by point-by-point convolution to expand the number of channels, followed by k*k heterogeneous depthwise separable convolution operations, and finally point-by-point convolution is used to reduce the number of channels and compensate for the information loss caused by the depthwise separable convolution. During the training of the inverted residual bottleneck, the network runs n depthwise convolutions of different sizes in parallel. This module expands the perception range and retains the information of small objects without incurring additional inference costs by parallelizing large kernel convolutions with several small kernel convolutions.

[0096] like Figure 4 As shown, the local-global hybrid self-attention mechanism includes three input branches. The first input branch passes through a linear projection layer, layer normalization, one-dimensional convolution, layer normalization, and compressed excitation attention to output global information. The second input branch passes through a linear projection layer, layer normalization, multi-head self-attention, layer normalization, and compressed excitation attention to output local information. The global information, local information, and original input are jointly input into an adaptive hybrid global and local information module to form a mixed output. The mixed output is transformed by a linear projection layer and added to the original input to form a residual connection, and the output information is processed by layer normalization.

[0097] The local-global mixed self-attention mechanism jointly models the user's long-term and short-term preferences by combining local convolutional layers and global self-attention layers. The local preferences (captured by CNN) and global preferences (captured by Transformer) are combined to more comprehensively model the user's dynamic preferences; it adaptively mixes global and local information, decouples the fusion process in different layers, improves the expressiveness, and adaptively aggregates long-term and short-term preferences (the mixed importance of local and global dependency modules can be adjusted according to the user's personalized needs); compressed incentive attention is used to replace the softmax operation, allowing multiple related items to be considered at the same time, enhancing the model's expressiveness (allowing the model to focus on multiple highly related objects at the same time, instead of focusing on only a single object like traditional softmax).

[0098] Compressed Incentive Attention: Takes an input tensor of shape [B, N, D], where B is the batch size, N is the sequence length, and D is the feature dimension.

[0099] First, the average value of the input tensor along the feature dimension D is calculated to obtain the average value of the tensor with shape [B, N, 1].

[0100] Transpose the dimensions of the input tensor mean to obtain a tensor of shape [B, 1, N].

[0101] The transposed tensor is transformed through the first linear layer to obtain a tensor hidden layer state with a shape of [B, 1, N / r], where N / r represents the compressed dimension.

[0102] Apply an activation function (ReLU) to further nonlinearly activate the hidden layer states.

[0103] The activated tensor is transformed back to the original sequence length [B, 1, N] through a second linear layer.

[0104] Apply the Sigmoid function to generate the attention weight tensor attention score, with a shape of [B, 1, N].

[0105] The attention scores are transposed again so that they can be multiplied with the original input tensor, resulting in a tensor of shape [B, N, 1].

[0106] Adaptively mix global and local information, receiving three input tensors: model input, global information, and local information.

[0107] Calculates the average of the input tensor along the feature dimension for generating adaptive weights.

[0108] The adaptive weight alpha is obtained through a linear projection layer and compressed excitation attention and expanded to a [B, 1,1] shape (where B represents the input sample batch size).

[0109] beta is set to 1-alpha, representing the remaining proportion.

[0110] According to these two weights, the global information and the local information are mixed together to form a mixed output.

[0111] The mixed output is transformed by the linear projection layer and then added to the original input to form a residual connection.

[0112] The final output is processed through a dropout layer and layer normalization.

[0113] Dynamically adjust the weight between global and local information. Use residual connections to keep gradients flowing and reduce the risk of gradient vanishing. Enhance the generalization ability of the model through layer normalization and dropout.

[0114] like Figure 5 As shown, the input of the relative position enhanced self-attention mechanism enters the query matrix linear projection layer, the key matrix linear projection layer, the value matrix linear projection layer, and the parameter matrix linear projection layer respectively. The outputs of the matrix linear projection layer and the key matrix linear projection layer are matrix multiplied and added to the relative position of the bias. After being processed by the softmax function, they are matrix multiplied with the output of the value matrix linear projection layer to form the first output. The output of the parameter matrix linear projection layer acts on the tokens depthwise separable convolution, and is linearly projected at the tokens level to form the second output. The first output is added to the second output to obtain the final output.

[0115] Relative Position Enhanced Self-Attention Mechanism, Importance of Position Information, For computer vision tasks, pixels or patches in an image are regarded as tokens, and their position information is crucial for constructing image semantics. A key feature of the self-attention mechanism is that it is insensitive to the order of the tokens, which means that it cannot directly utilize the position information of the tokens. Introducing relative position bias: Methods such as Swin Transformer guide the model to learn position information and content-based attention by introducing an additional relative position bias in the attention matrix. The relative position enhanced self-attention mechanism further develops this idea by establishing an independent value representation space to capture position dependencies. Mathematical formula of the relative position enhanced self-attention mechanism: The relative position enhanced self-attention mechanism transforms the token into a new representation by introducing a parameter matrix Vrel in order to mix information by relative position. The relative position enhanced self-attention mechanism uses a weight tensor brel, which is indexed by relative position. The relative position enhanced self-attention mechanism also retains the relative position bias B to give the model greater flexibility. Limiting the maximum absolute distance: In order to enhance the generalization ability of the model and simplify the implementation, the relative position enhanced self-attention mechanism limits the maximum absolute relative position distance s on each axis. After doing so, the computation of the relative position enhanced self-attention mechanism can be simplified to a token-based convolution with kernel size (2s+1)*(2s+1).

[0116] The feature fusion neck includes three layers of branches, each of which includes a shallow feature interaction module, a cross-stage blocking module, a deep feature interaction module, and a spatial context perception module in sequence. The shallow feature interaction module receives the first features of the same layer and the previous layer, and outputs the shallow features to the cross-stage blocking module. The cross-stage blocking module distributes the shallow features to the front deep feature interaction modules and the rear shallow feature interaction module of the previous layer. The deep feature interaction module outputs the deep features to the spatial context perception module. The ternary scale sequence fusion module obtains the first feature of the same layer of each branch, and after feature splicing, it is input into the first-layer spatial context perception module. The spatial context perception module outputs the global context feature to the rear next-layer deep feature interaction module and to the prediction head.

[0117] After each efficient layer aggregation module, there is a shallow feature interaction module responsible for cross-scale information exchange and fusion. The shallow feature interaction module is followed by the cross-stage block module, which is a multi-branch structure that allows interaction between features at different levels. The deep feature interaction module utilizes additional deep information to enhance feature representation. The spatial context-aware module may capture spatial relationships and generate richer feature representations.

[0118] like Figure 6As shown, the shallow feature interaction module receives the first features of the previous layer and the same layer, and the shallow features of the next layer. The first features of the previous layer are subjected to depth-separable convolution to form deep information, which is stacked with the first features of the same layer and the shallow features of the next layer to output shallow features.

[0119] Shallow feature interaction module, accurate positioning depends on detailed edge information obtained from the shallow network, while accurate classification requires a deeper network to capture coarse-grained information. An effective feature pyramid network should support the full and sufficient fusion of shallow and deep network information flows; retaining shallow spatial information in the backbone network is crucial to enhancing the detection ability of smaller targets. However, the information provided by the backbone network is relatively basic and easily disturbed. Therefore, the shallow information is integrated into the deep network as an auxiliary branch to ensure the stability of subsequent layer learning; the main goal is to integrate deep information with high-resolution shallow features from the same layer and the next layer in the backbone network, aiming to retain rich positioning details to enhance the spatial performance of the network; use 1*1 convolution to control the number of channels in the shallow information to ensure that it accounts for a small proportion in the splicing operation and does not affect subsequent learning. The shallow backbone information is maintained through bidirectional connections, which enhances the network's ability to detect small targets.

[0120] like Figure 7 As shown in the figure, the deep feature interaction module receives the shallow features of the previous layer, the same layer, and the next layer, as well as the global context features of the previous layer, the shallow features of the previous layer, the shallow features of the next layer, and the global context features of the previous layer, which are respectively subjected to deep separable convolution and then stacked with the shallow features of the same layer to output deep features.

[0121] The deep feature interaction module performs multi-scale information fusion at a deeper level to further improve the interactive use of feature layer information. These connections span the shallow high-resolution layer, the shallow low-resolution layer, the shallow layer of the same level, and the front spatial context perception to aggregate information. The gradient information of the output layer is enriched through multi-directional connections.

[0122] like Figure 8 As shown in the figure, the ternary scale sequence fusion module includes three layers of branches, which receive three layers of first features of different scales respectively, and each passes through a convolution module to adjust the channel. The first layer of branches is downsampled by maximum pooling and average pooling, and then processed by the convolution module. The third layer of branches is upsampled by the nearest neighbor interpolation method, and then processed by the convolution module. The three layers of branch features are stacked and then processed by the three-dimensional convolution module to output the spliced ​​features.

[0123] Ternary Scale Sequence Fusion Module,In order to identify densely overlapping small objects, the shape or appearance changes at different scales can be referenced and compared by enlarging the image. Since different feature layers of the backbone network have different sizes, the traditional feature pyramid fusion mechanism only upsamples the feature maps of small sizes and then splits or adds them to the features of the previous layer, ignoring the rich detail information of the large-size feature layer. The ternary scale sequence fusion module can split the features of large, medium, and small sizes, add the feature maps of large sizes, and enlarge the features to improve the detailed feature information. It can better combine the high-dimensional information of the deep feature map and the detailed information of the shallow feature map. The scale space is constructed along the scale axis of the image, which not only represents a scale, but also represents the range of various scales that an object may have. Feature maps of different scales are stacked horizontally, and their scale sequence features are extracted using 3D convolution. The high-resolution feature map contains most of the information that is crucial for the detection of small objects. Before feature encoding, the number of feature channels should be adjusted to keep it consistent with the main scale features. After the convolution module processes the large-size feature map, its number of channels is adjusted to 1C, and then a hybrid structure of maximum pooling and average pooling is used for downsampling, which helps to retain the effectiveness and diversity of high-resolution features and cell images. For small-size feature maps, the convolution module is also used to adjust the number of channels, and then the nearest neighbor interpolation method is used for upsampling. This helps to maintain the richness of local features of low-resolution images and prevent the loss of feature information of small targets. Finally, the three feature maps of the same size, large, medium, and small, are convolved once, then spliced ​​in the channel dimension, and finally three-dimensional convolution, batch normalization, and activation function are performed.

[0124] like Fig. 9 As shown in the figure, the context-aware module includes three branches, which perform global average pooling, global maximum pooling, and point-by-point convolution on the input features respectively, and then perform softmax function, matrix multiplication, and point-by-point convolution after global average pooling and global maximum pooling. The point-by-point convolution is processed by softmax function, matrix multiplication, and point-by-point convolution, and finally perform Hadamard product together to output global context features.

[0125] The spatial context-aware module is designed to model global context information across channels and spatial dimensions and suppress useless background features. The structure consists of three branches, using global average pooling (GAP) and global maximum pooling (GMP) to integrate global information. The functions are: extracting contextual relationships between space and channels. Enhance the feature representation of small objects and reduce background confusion. Implement linear transformations that simplify feature maps using 1x1 convolutions. Calculate channel and spatial context information through matrix multiplication. Hadamard product operation, output feature maps containing global context.

[0126] Training of person recognition model Training data production: Find L videos containing pedestrians in different locations, different times and different weather conditions in the surveillance video. Vi represents the i-th video. There are Ni video images in Vi. Select Mi video images from Ni video images as training and test images.

[0127] Data enhancement of training data images: brightness enhancement, contrast enhancement, image rotation, image flipping, affine change image expansion, shear change image expansion, HSV data enhancement, translation expansion; eight ways to perform data enhancement.

[0128] Hyperparameters for network training Input Resolution Default: The default input resolution for the network is 640x640.

[0129] Setting suggestions: Adjust according to actual needs. Increasing the input resolution can improve the accuracy of the model, but it will also increase the amount of calculation and inference time. If the application scenario has high requirements for speed, you can appropriately reduce the input resolution.

[0130] Batch size: 256 Influencing factors: Batch size affects the training speed and memory consumption of the model. Setting suggestions: Choose an appropriate batch size based on the graphics card memory size and actual needs. A larger batch size can speed up training, but it will also take up more graphics memory.

[0131] Learning Rate Initial learning rate: Usually 0.01 when using the SGD optimizer and 0.001 when using the Adam optimizer.

[0132] Scheduling strategy: You can use a learning rate decay strategy, such as cosine annealing, to adjust the learning rate.

[0133] Setting suggestions: Adjust the initial learning rate according to the specific problem, dataset, and model size, and observe the changes in the loss curve during training and adjust the learning rate in time.

[0134] Momentum Default value: In the SGD optimizer, YOLOv5 usually uses a higher momentum value, such as 0.937, to improve training speed and stability. Setting suggestion: The choice of momentum value should be adjusted according to the actual situation. A larger momentum can speed up parameter updates, but it may also cause overfitting.

[0135] Weight Decay Function: A regularization method to prevent overfitting of the model. Default value: The commonly used weight decay value in the network is 0.0005.

[0136] Setting suggestions: Adjust the weight decay value according to the complexity of the model and the characteristics of the dataset. Larger values ​​increase the regularization strength, but may also cause the model to underfit.

[0137] Training Iterations (Epochs) Definition: The number of training iterations refers to the number of times the entire training set is passed through the model's forward propagation and backpropagation to update the parameters.

[0138] Setting suggestions: Choose an appropriate number of iterations by observing the loss values ​​and indicator changes on the training and validation sets. Too few iterations will lead to underfitting of the model, while too many iterations may lead to overfitting of the model.

[0139] Data Augmentation Function: Expand the data set by randomly transforming the training data to improve the generalization ability and robustness of the model.

[0140] Setting suggestions: There are many data augmentation methods built into the network, such as flipping, scaling, random cropping, etc. Choose the appropriate data augmentation method based on the characteristics of the specific problem and dataset.

[0141] Other hyperparameters Model depth and width: The network introduces depth_multiple and width_multiple coefficients to obtain models of different sizes. Choose the appropriate model size according to the needs of the application scenario.

[0142] Loss function weight: The loss functions in the network include bounding box loss (box loss), classification loss (clsloss), and object existence loss (obj loss). The importance of different tasks can be balanced by adjusting the weights of these losses.

[0143] IOU threshold: The IOU threshold used to determine the bounding box of positive and negative samples. A higher IOU threshold can better filter out negative samples and inaccurate positive samples, but it may also lead to a decrease in the number of positive samples.

[0144] Model function: Extract continuous visual information from the video data source in real time, and then use a carefully trained and optimized neural network to deeply analyze each frame of the image. In particular, the model is designed to accurately identify and locate the head targets in the picture. It can not only accurately mark the location of each head (precisely defined by the bounding box), but also evaluate the confidence of these detection results, that is, the degree to which the model is sure of the correctness of its judgment. Through this process, users can be provided with intuitive and reliable head detection and tracking results, enabling a variety of application scenarios, such as counting people.

[0145] Model iteration: First, we carefully collected a batch of new data sets that cover a wide range of scenarios and conditions, providing rich materials for further training of the model. Then, we used the current model to conduct a detailed test on these data, and divided them into four categories based on the test results: images that clearly contain the target, images that are falsely reported as targets, images that do not contain targets but may not be detected due to conditions, and images that do not contain any targets.

[0146] Based on this classification, targeted strategies are adopted to process various types of images. In particular, images that are falsely reported as targets are regarded as valuable negative sample resources, which help the model learn how to distinguish true targets from background noise. Images that contain targets but are not detected by the current model are regarded as key training samples, which directly point out the room for improvement of model performance.

[0147] Next, we focus on fine-grained data annotation of the images without detected targets to ensure that each target is accurately marked. At the same time, we use a variety of data enhancement techniques, such as rotation, scaling, color adjustment, etc., to enrich the diversity of training samples and improve the generalization ability of the model.

[0148] Based on the original model, we use these newly annotated and enhanced data to retrain the model, and by continuously optimizing learning parameters and adjusting the network structure, we strive to significantly improve the detection accuracy while maintaining the model efficiency. After training, we test the model and compare the detection results of the new and old models to evaluate the improvement of the model performance.

[0149] A green base station energy-saving method based on the fusion of self-attention and convolutional deep learning, and an algorithm code business logic implementation method for predicting the number of people. First, the interface accepts parameters such as the camera address, algorithm type, and callback address; second, the interface will start a new process, start capturing image frames from the camera's video stream, and store them in redis, and notify the listening program at the same time; third, the listening program receives the notification, retrieves the image frame from redis, and starts creating an algorithm instance, and calls the algorithm instance to start analyzing the image frame; fourth, the algorithm instance analyzes the image frame according to the business logic, stores the analysis results in redis, and notifies the listening program at the same time; fifth, the listening program receives the notification, retrieves the results, and submits the analysis results to the business interface (callback).

[0150] The person recognition model of the present invention can segment large, medium and small-sized features by adopting a ternary scale sequence fusion model, add a large-sized feature map, and amplify the features to improve the detailed feature information, and can better combine the high-dimensional information of the deep feature map and the detailed information of the shallow feature map; adopt a relative position enhanced self-attention mechanism to make up for the position blind spot of self-attention, and the model has a powerful ability to capture dependencies from content and position; adopt a spatial context perception module to extract the contextual relationship between space and channels, enhance the feature representation of small targets, and reduce background confusion; adopt a local-global mixed self-attention mechanism to combine local preferences with global preferences to more comprehensively model the user's dynamic preferences.

[0151] The specific embodiments described herein are merely examples of the spirit of the present invention. Those skilled in the art may make various modifications or additions to the specific embodiments described or replace them in similar ways, but they will not deviate from the spirit of the present invention or exceed the scope defined by the appended claims.

Claims

1. A green base station energy-saving method based on the fusion of self-attention and convolutional deep learning, characterized in that: The following steps are involved: S1. Periodic scanning time node acquisition monitoring first video image; S2. Using a person recognition model that combines self-attention and convolutional deep learning to perform person recognition and counting on the first video image; S3. Analyze the movement trend of people and dynamically count the number of people in the monitoring area; S4. Calculate the number of users in the monitoring area and activate the corresponding base station capacity resources according to the number of users.

2. The green base station energy saving method based on the fusion of self-attention and convolutional deep learning according to claim 1 is characterized in that The person recognition model integrating self-attention and convolutional deep learning includes: The input image is converted into feature maps of multiple scale spaces through the backbone network, and the first features of each scale space are extracted and output respectively; Through the feature fusion neck, shallow feature interaction and deep feature interaction are performed on the first features of each scale space, and the first features of each scale space are spliced. According to the deep interaction and splicing features, context fusion is performed to output the second features of each scale space respectively; Each second feature input prediction head generates a multi-target output.

3. The green base station energy saving method based on the fusion of self-attention and convolutional deep learning according to claim 2 is characterized in that: The backbone network includes three efficient layer aggregation modules connected in sequence. The efficient layer aggregation modules extract features and output first features of different scales. The lowest efficient layer aggregation module obtains feature maps through local-global hybrid self-attention mechanism and relative position enhanced self-attention mechanism, and stacks them with feature maps of different scales on the upper layer of the feature map to output the first feature.

4. The green base station energy saving method based on the fusion of self-attention and convolutional deep learning according to claim 3 is characterized in that: The efficient layer aggregation module performs point-by-point convolution and feature separation operations on the input information, and after the operation, it is divided into a first branch and a second branch. The first branch enters the feature stack after N inverted residual bottlenecks, and the second branch retains the original information and enters the feature stack. Finally, point-by-point convolution is performed and the information is output; The local-global hybrid self-attention mechanism includes three input branches. The first input branch outputs global information after passing through a linear projection layer, layer normalization, one-dimensional convolution, layer normalization, and compressed excitation attention. The second input branch outputs local information after passing through a linear projection layer, layer normalization, multi-head self-attention, layer normalization, and compressed excitation attention. The global information, local information, and original input are input into an adaptive hybrid global and local information module to form a hybrid output. The hybrid output is transformed by a linear projection layer and then added to the original input to form a residual connection. The output information is processed by layer normalization. The relative position enhanced self-attention mechanism inputs enter the query matrix linear projection layer, the key matrix linear projection layer, the value matrix linear projection layer, and the parameter matrix linear projection layer respectively. The outputs of the matrix linear projection layer and the key matrix linear projection layer are matrix multiplied and added to the relative position of the bias. After being processed by the softmax function, they are matrix multiplied with the output of the value matrix linear projection layer to form a first output. The output of the parameter matrix linear projection layer acts on the tokens depth-separable convolution, and is linearly projected at the tokens level to form a second output. The first output is added to the second output to obtain the final output.

5. The green base station energy saving method based on the fusion of self-attention and convolutional deep learning according to claim 2 is characterized in that: The feature fusion neck includes three layers of branches, each layer of branches includes a shallow feature interaction module, a cross-stage blocking module, a deep feature interaction module, and a spatial context perception module in sequence. The shallow feature interaction module receives the first features of the same layer and the previous layer, and outputs the shallow features to the cross-stage blocking module. The cross-stage blocking module distributes the shallow features to the front deep feature interaction modules and the rear upper shallow feature interaction module. The deep feature interaction module outputs the deep features to the spatial context perception module. The ternary scale sequence fusion module obtains the first features of the same layer of each branch, and after feature splicing, it is input into the first layer of spatial context perception module. The spatial context perception module outputs the global context features to the rear next layer of deep feature interaction module and to the prediction head.

6. The green base station energy saving method based on the fusion of self-attention and convolutional deep learning according to claim 5 is characterized by: The shallow feature interaction module receives the first features of the previous layer and the same layer, and the shallow features of the next layer. The first features of the previous layer are subjected to depth-separable convolution to form deep information, which is stacked with the first features of the same layer and the shallow features of the next layer to output shallow features; The deep feature interaction module receives the shallow features of the previous layer, the same layer, and the next layer, and the global context features of the previous layer, the shallow features of the previous layer, the shallow features of the next layer, and the global context features of the previous layer, which are respectively subjected to depth-separable convolution and then stacked with the shallow features of the same layer to output deep features; The ternary scale sequence fusion module includes three layers of branches, which respectively receive three layers of first features of different scales, each of which passes through a convolution module to adjust the channel. The first layer of branches is downsampled by maximum pooling and average pooling, and then processed by the convolution module. The third layer of branches is upsampled by the nearest neighbor interpolation method, and then processed by the convolution module. The three layers of branch features are then stacked and processed by the three-dimensional convolution module to output the spliced ​​features. The context-aware module includes three branches, which respectively perform global average pooling, global maximum pooling, and point-by-point convolution on the input features, and then perform softmax function, matrix multiplication, and point-by-point convolution on the global average pooling and global maximum pooling. The point-by-point convolution is processed by softmax function, matrix multiplication, and point-by-point convolution, and finally perform Hadamard product together to output global context features.

7. The green base station energy saving method based on the fusion of self-attention and convolutional deep learning according to claim 2 is characterized by: The step S3 specifically includes: S31. Analyze the motion state of the person through deep learning, determine the moving direction of the person in motion, calculate the moving trajectory of the person, and determine whether the person is about to exceed the current video image range; S32. If not, count the number of people in each monitoring area and proceed to step S4; If yes, determine the range where the person is about to appear based on the task movement trajectory, obtain the second video image of the monitoring after the set delay time, perform person recognition and counting on the second video image, count the number of people in each monitoring area, and enter step S4, while judging whether the current time exceeds the current scanning cycle; S33. If yes, enter the next scanning cycle and return to step S1; If not, return to step S31.

8. The green base station energy saving method based on the fusion of self-attention and convolutional deep learning according to claim 1 or 7, characterized in that: After step S2, the method further includes: Determine whether there is a person in the character recognition result. If yes, go to step S3; if no, notify the 5G base station to enter the waiting, sleep or deep sleep state.

9. The green base station energy saving method based on the fusion of self-attention and convolutional deep learning according to claim 1, 2 or 7, characterized in that: The calculation of the number of users in the monitoring area specifically includes: Obtain the operator user penetration rate in this business area and calculate the number of operator users in each monitoring area; Get the operator's 5G user ratio and calculate the number of operators' 5G users in each monitoring area.

10. The green base station energy saving method based on the fusion of self-attention and convolutional deep learning according to claim 1 or 2, characterized in that It also includes real-time adjustment steps, including: Calculate the expected capacity requirements of users based on the number of users in the monitoring area, and obtain the actual capacity requirements of users in the monitoring area; Compare the user's expected capacity demand with the user's actual capacity demand, and adjust the base station capacity resources in real time according to the user's actual capacity demand.

Images