A target detection method based on a shared backbone network and a gating unit

Abstract

This invention discloses a target detection method based on a shared backbone network and gating units, belonging to the field of computer vision target detection technology. The method includes: acquiring image data to be tested; inputting the image data to be tested into a target detection model, the target detection model including a shared backbone network, a neck network, gating units, and multiple candidate prediction heads; extracting features through the shared backbone network to obtain an initial feature map; performing multi-scale feature fusion through the neck network to obtain prediction feature maps of various scales; determining activation values ​​based on the prediction feature maps of various scales through gating units; selecting a target prediction head from multiple candidate prediction heads based on the activation values; and inputting the prediction feature maps of various scales into the target prediction head to obtain the target detection result. This invention enables a single target detection model to adapt to different camera perspectives, eliminating the need to maintain multiple independent models, thus reducing computational overhead and maintenance costs.

Description

Technical Field

[0001] This invention relates to the field of computer vision target detection technology, and in particular to a target detection method based on a shared backbone network and gating units. Background Technology

[0002] In the field of intelligent transportation, highway service area parking scenarios typically require target detection methods using high-pole cameras to detect vehicles in real time to determine parking space occupancy status. In practical applications, due to varying camera installation angles, the perspectives of the vehicles captured vary significantly. For example, from a standard perspective, the camera angle to the ground is approximately 45 degrees, and the visible area of ​​the target vehicle includes the front / rear, sides, and roof. However, from a top-down perspective, the camera angle to the ground is 90 degrees, and only the roof area is visible. This change in vehicle characteristics due to the varying perspectives poses a significant challenge to the detection effectiveness and generalization performance of target detection methods.

[0003] Currently, there are two types of object detection solutions for the above scenarios: one is to use a general 2D object detection method trained for different viewpoints, and the other is to use a target detection method with rotation angle trained for camera overhead viewpoints or drone aerial viewpoints.

[0004] The two existing object detection schemes mentioned above typically suffer from the following technical drawbacks. In general 2D object detection methods, the algorithm predicts a horizontal rectangular bounding box for the target, meaning the box's length and width are parallel to the X or Y axis of the pixel coordinate system. This horizontal bounding box cannot accurately fit the target contour, and it contains a large amount of redundant pixel information, interfering with subsequent feature extraction or business logic judgment. When facing significantly different shooting angles, the detection performance and generalization capabilities of the object detection algorithm usually cannot meet business requirements. In object detection methods with rotation angles, the algorithm predicts the target's rotation angle based on the conventional 2D object detection algorithm for a top-down view, outputting a rectangle with the rotation angle. This operation can accurately locate targets facing any orientation, significantly reducing background redundant pixels and improving positioning accuracy. However, limited by the top-down view, facing complex and varied camera angles usually requires maintaining multiple network models, including both the general 2D object detection method and the object detection method with rotation angles, leading to high data maintenance and development costs.

[0005] Therefore, how to maintain detection accuracy while adapting a single target detection method to vehicle target detection tasks under different camera perspectives, and reduce model maintenance costs and computational overhead in the implementation of the method, is a technical problem that urgently needs to be solved in the field of vehicle target detection technology in highway service area parking scenarios. Summary of the Invention

[0006] The technical problem to be solved by this invention is to address the shortcomings of existing technologies, specifically the difficulty of using a single model to adapt to different camera shooting angles for target detection. Specifically, this invention provides a target detection method based on a shared backbone network and gating units, as detailed below: 1) In a first aspect, the present invention provides a target detection method based on a shared backbone network and gating units, the specific technical solution of which is as follows: S1, acquire the image data to be tested, which is obtained by an image acquisition device capturing images of the target area; S2, the image data to be tested is input into the target detection model, the target detection model includes a shared backbone network, a neck network, a gating unit and multiple candidate prediction heads; S3, feature extraction is performed on the image data to be tested through the shared backbone network to obtain an initial feature map; S4, Multi-scale feature fusion is performed on the initial feature map through the neck network to obtain prediction feature maps of various different scales; S5, the gating unit determines the activation value of the image data to be tested based on the predicted feature maps of various different scales; S6, Select a target prediction head from the plurality of candidate prediction heads according to the activation value; S7, input the prediction feature maps of various different scales into the target prediction head to obtain the target detection result corresponding to the image data to be tested.

[0007] The beneficial effects of the target detection method based on a shared backbone network and gating units provided by this invention are as follows: Multi-scale feature extraction and fusion are achieved by sharing the backbone and neck networks. The gating unit calculates the activation value based on the multi-scale predicted feature map and selects a target prediction head from multiple candidate prediction heads to output the detection result. This allows a single target detection model to adaptively select the appropriate prediction head for output based on different viewpoints of the input image, eliminating the need to maintain multiple independent network models for different viewpoints, thus reducing data maintenance and development costs. At the same time, since different prediction heads share the same weight parameters of the backbone and neck networks, computational overhead is significantly reduced. By selecting a prediction head adapted to the viewpoint to output detection results in a specific format, background redundancy information in the detection results can be reduced, improving localization accuracy.

[0008] Based on the above solution, the present invention can be further improved as follows.

[0009] Furthermore, the gating unit includes a 1×1 convolution module, a global average pooling module, a feature fusion module, and a nonlinear activation module; Determining the activation value of the image data to be tested specifically includes: The number of channels in the predicted feature map at each scale is mapped to the same dimension via the 1×1 convolution module to obtain a multi-scale feature map with the same channel dimension. Each scale feature map of the multi-scale feature map with the same channel dimension is compressed along the spatial dimension by the global average pooling module to obtain multiple scale compression vectors. The multiple scale-compressed vectors are concatenated along the height direction by the feature fusion module to obtain a fused vector; The fusion vector is numerically compressed using the global average pooling module and the nonlinear activation module to obtain the activation value.

[0010] Furthermore, the plurality of candidate prediction heads includes a first prediction head and a second prediction head; The first prediction head is used to output a two-dimensional bounding box prediction result, which includes the center point coordinates, width, height, and foreground probability of the target object. The second prediction head is used to output keypoint prediction results, which include the coordinates of multiple keypoints of the target object and the foreground probability.

[0011] Furthermore, it also includes training the target detection model: inputting training image data labeled with different viewpoint classification labels into the target detection model, and using the gated unit activation value loss function to perform parameter gradient backpropagation and parameter convergence of the target detection model; The gating unit activation value loss function is calculated based on the classification label of each training image data and the activation value corresponding to each training image data; the different perspectives include a first perspective and a second perspective.

[0012] Furthermore, the total loss function of the target detection model is divided into a first-view total loss function and a second-view total loss function; The first viewpoint total loss function includes target foreground or background probability loss, target classification loss, two-dimensional bounding box regression loss, and the loss value corresponding to the gating unit activation value loss function; The second perspective total loss function includes target foreground or background probability loss, target classification loss, keypoint loss, and the loss value corresponding to the gating unit activation value loss function.

[0013] 2) In a second aspect, the present invention also provides a target detection system based on a shared backbone network and a gating unit, the specific technical solution of which is as follows: an acquisition module, an input module, an extraction module, a fusion module, a calculation module, a selection module, and an output module; The acquisition module is used to acquire the image data to be tested, which is obtained by the image acquisition device from the target area. The input module is used to input the image data to be tested into the target detection model, which includes a shared backbone network, a neck network, a gating unit, and multiple candidate prediction heads. The extraction module is used to extract features from the image data to be tested through the shared backbone network to obtain an initial feature map. The fusion module is used to perform multi-scale feature fusion on the initial feature map through the neck network to obtain prediction feature maps of various different scales; The calculation module is used to determine the activation value of the image data to be tested based on the predicted feature maps of various different scales through the gating unit; The selection module is used to select a target prediction head from the plurality of candidate prediction heads according to the activation value; The output module is used to input the prediction feature maps of various scales into the target prediction head to obtain the target detection result corresponding to the image data to be tested.

[0014] Based on the above solution, the present invention can be further improved as follows.

[0015] Furthermore, the gating unit includes a 1×1 convolution module, a global average pooling module, a feature fusion module, and a nonlinear activation module; Determining the activation value of the image data to be tested specifically includes: The number of channels in the predicted feature map at each scale is mapped to the same dimension via the 1×1 convolution module to obtain a multi-scale feature map with the same channel dimension. Each scale feature map of the multi-scale feature map with the same channel dimension is compressed along the spatial dimension by the global average pooling module to obtain multiple scale compression vectors. The multiple scale-compressed vectors are concatenated along the height direction by the feature fusion module to obtain a fused vector; The fusion vector is numerically compressed using the global average pooling module and the nonlinear activation module to obtain the activation value.

[0016] Furthermore, the plurality of candidate prediction heads includes a first prediction head and a second prediction head; The first prediction head is used to output a two-dimensional bounding box prediction result, which includes the center point coordinates, width, height, and foreground probability of the target object. The second prediction head is used to output keypoint prediction results, which include the coordinates of multiple keypoints of the target object and the foreground probability.

[0017] Furthermore, it also includes training the target detection model: inputting training image data labeled with different viewpoint classification labels into the target detection model, and using the gated unit activation value loss function to perform parameter gradient backpropagation and parameter convergence of the target detection model; The gating unit activation value loss function is calculated based on the classification label of each training image data and the activation value corresponding to each training image data; the different perspectives include a first perspective and a second perspective.

[0018] Furthermore, the total loss function of the target detection model is divided into a first-view total loss function and a second-view total loss function; The first viewpoint total loss function includes target foreground or background probability loss, target classification loss, two-dimensional bounding box regression loss, and the loss value corresponding to the gating unit activation value loss function; The second perspective total loss function includes target foreground or background probability loss, target classification loss, keypoint loss, and the loss value corresponding to the gating unit activation value loss function.

[0019] 3) In a third aspect, the present invention also provides a computer device, the computer device including a processor coupled to a memory, the memory storing at least one computer program, the at least one computer program being loaded and executed by the processor to enable the computer device to implement any of the above methods.

[0020] 4) In a fourth aspect, the present invention also provides a computer-readable storage medium storing at least one computer program, which is loaded and executed by a processor to enable a computer to implement any of the above methods.

[0021] It should be noted that the beneficial effects of the technical solutions of the second to fourth aspects of the present invention and their corresponding possible implementations can be found in the above description of the technical effects of the first aspect and its corresponding possible implementations, and will not be repeated here. Attached Figure Description

[0022] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a flowchart illustrating the steps of a target detection method based on a shared backbone network and gating units according to an embodiment of the present invention. Figure 2 This is a schematic diagram of the overall structure of a target detection model according to an embodiment of the present invention, which is a target detection method based on a shared backbone network and a gating unit. Figure 3 This is a schematic diagram of the gating unit network structure of a target detection method based on a shared backbone network and gating units according to an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of a computer device according to an embodiment of the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0024] like Figure 1 As shown, an embodiment of the present invention provides a target detection method based on a shared backbone network and gating units, comprising the following steps: S1, acquire the image data to be tested. The image data to be tested is obtained by the image acquisition device capturing images of the target area. The specific implementation method is as follows: Image acquisition equipment refers to hardware devices used to capture images of a target area to obtain image data to be tested. In this specific implementation, it is a high-pole camera, but other devices with image capture capabilities, such as surveillance cameras, drone-mounted cameras, and handheld cameras, are also applicable. The target area refers to the area in a highway service area parking lot where the occupancy status of parking spaces needs to be monitored. In this specific implementation, it is the highway service area parking lot area, but other scenarios requiring target detection, such as urban traffic intersections, internal roads of industrial parks, and indoor parking lots, are also applicable. The image data to be tested is the original image captured by the high-pole camera of the highway service area parking lot area. In the highway service area parking lot scenario, a high-pole camera is used as the image acquisition device. The installation height of the high-pole camera is no less than 6 meters, and the installation angle is between 30 and 90 degrees. The high-pole camera captures images of vehicles within the parking lot area in real time, obtaining image data containing vehicles as the image data to be tested. The shooting scene should include rich lighting and weather conditions, such as daytime, nighttime, sunny days, and rainy / snowy days, to improve the generalization ability of subsequent model processing.

[0025] The beneficial effects of this step are: acquiring vehicle image data of highway service area parking lot scenes through high-pole cameras provides a data source for target detection, and improving the scene adaptability of the detection method by acquiring images covering rich lighting and weather conditions.

[0026] S2, the image data to be tested is input into the object detection model, which includes a shared backbone network, a neck network, gating units, and multiple candidate prediction heads. The specific implementation is as follows: An object detection model is built using a deep learning framework (such as PyTorch). This model consists of four components: a shared backbone network, a neck network, gating units, and multiple candidate prediction heads. The overall structure of the object detection model is as follows: Figure 2 As shown. The shared backbone network refers to the network structure in the object detection model used to extract the basic features of the target object from the input image. In specific implementations, a residual network (ResNet50) convolutional neural network (CNN) is used. Other convolutional neural network structures with feature extraction functions, such as ResNet101, MobileNet, and EfficientNet, are also applicable. The target object refers to the physical object that the object detection model needs to identify and locate from the image data to be tested. In specific implementations, the target object is a vehicle in a highway service area parking lot scene, including different types such as cars, trucks, and buses. The neck network refers to the network structure in the object detection model located after the shared backbone network and before the gating unit and the prediction head. It is used to perform multi-scale fusion processing on the feature map output by the shared backbone network. In specific implementations, a feature pyramid network (FPN) structure is used. Other network structures with multi-scale feature fusion functions, such as path aggregation network (PANet) and bidirectional feature pyramid network (BiFPN), are also applicable. The gating unit is a network structure in the object detection model used to calculate activation values ​​based on the input feature map to control the selection of the prediction head. It consists of a 1×1 convolution module, a global average pooling module, a feature fusion module, and a non-linear activation module. The activation value output by the gating unit determines which candidate prediction head to use for the object detection result output. A candidate prediction head is a network structure in the object detection model used to predict the prediction feature map output by the neck network. Multiple candidate prediction heads correspond to different output formats and application scenarios. In a specific implementation, the candidate prediction head includes a first prediction head for outputting the 2D bounding box prediction result and a second prediction head for outputting the keypoint prediction result. However, the number of candidate prediction heads is not limited to two and can be expanded to three or more depending on the actual application scenario. The image data to be tested obtained in S1 is input to the object detection model and processed by the above four components (see S3-S7 below for details).

[0027] The beneficial effects of this step are: by constructing an object detection model that includes a shared backbone network, a neck network, gating units, and multiple candidate prediction heads, a unified model architecture foundation is provided for multi-view adaptive detection, reducing data maintenance costs and R&D costs, and reducing computational overhead.

[0028] S3 extracts features from the image data under test using a shared backbone network to obtain an initial feature map. The specific implementation is as follows: The shared backbone network employs the ResNet50 structure from residual convolutional neural networks. ResNet50 is a specific implementation of a residual network, containing 50 convolutional layers. It addresses the vanishing gradient problem in deep network training through residual connections and is widely used in image feature extraction tasks. The shared backbone network is used to extract texture, contour, and color features of the target object in the test image data. The test image data input from S2 is fed into the shared backbone network, and after being processed step-by-step through multiple convolutional and pooling layers of ResNet50, high-level semantic features are extracted from the test image data, outputting an initial feature map. The initial feature map retains the spatial location and semantic information of the target object and serves as the input to the neck network.

[0029] The beneficial effect of this step is that by using a shared backbone network to extract features from the image data under test, an initial feature map containing rich semantic information such as texture, contour and color is obtained, which provides a basic feature representation for multi-scale feature fusion.

[0030] S4, through the neck network, performs multi-scale feature fusion on the initial feature map to obtain prediction feature maps of various scales. The specific implementation is as follows: The neck network employs a feature pyramid network structure. The feature pyramid network is a network structure used for multi-scale feature fusion. The initial feature map output from the shared backbone network in S3 is input into the feature pyramid network. The feature pyramid network fuses the semantic information of the multi-scale predicted feature maps from the bottom up, resulting in three different scale predicted feature maps, denoted as the first predicted feature map F1, the second predicted feature map F2, and the third predicted feature map F3. The predicted feature map refers to the feature map output by the neck network after multi-scale feature fusion, retaining spatial location and semantic information at different scales, and serves as the input to the gating unit and the prediction head. The three predicted feature maps differ in spatial dimensions (height and width), corresponding to the feature maps output by the feature pyramid network at different levels, and are used to detect target objects of different sizes. The number of channels for F1, F2, and F3 is [missing information]. The height and width are respectively , as well as The three are all different in size. A feature map is a set of two-dimensional matrices output after each convolutional operation in a convolutional neural network. Each channel corresponds to a two-dimensional matrix, and each element in the matrix represents the feature response intensity at that location.

[0031] The beneficial effect of this step is that by performing multi-scale feature fusion on the initial feature map through the feature pyramid network structure, three different scales of predicted feature maps are obtained, which enables the gating unit to combine feature information of multiple scales to make viewpoint judgment, and at the same time enables the prediction head to have good detection effect on targets of different sizes.

[0032] S5 determines the activation values ​​of the image data to be tested based on predicted feature maps of various scales using a gating unit. The specific implementation is as follows: In a preferred embodiment Figure 3 This is a schematic diagram of the gating unit network structure, such as... Figure 3 As shown, the gating unit includes a 1×1 convolution module and a global average pooling module (corresponding to...). Figure 3 The network consists of three modules: a global average pooling module, a feature fusion module, and a non-linear activation module. The 1×1 convolution module uses 1×1 kernels to perform convolution operations, adjusting the number of channels in the feature map without changing its spatial size. In specific implementations, it maps the number of channels in three different scales of predicted feature maps to the same dimension. The global average pooling module calculates the average of all values ​​along the spatial dimension for each channel of the feature map, compressing the spatial information of each channel into a single value. In specific implementations, it compresses the feature map into a vector along the spatial dimension. The feature fusion module merges multiple feature vectors into a single feature vector. In specific implementations, it merges the three scale compressed vectors into a fused vector by concatenating them along the height direction. The non-linear activation module performs a non-linear transformation on the input values, introducing non-linear mapping capabilities. In specific implementations, it uses the Sigmoid activation function to compress the values ​​of the fused vector to between 0 and 1.

[0033] The activation value of the image data to be tested is determined by the gating unit, specifically including: The number of channels in the predicted feature map at each scale is mapped to the same dimension using a 1×1 convolutional module, resulting in multi-scale feature maps with the same channel dimension. Specifically, the three different scale predicted feature maps F1, F2, and F3 are each passed through a 1×1 convolutional module, mapping the number of channels in all three predicted feature maps to the same dimension. This yields multi-scale feature maps with the same channel dimension, meaning all three feature maps have the same number of channels. However, the spatial dimensions (height and width) remain unchanged, still being respectively , as well as .

[0034] Each scale feature map in the multi-scale feature maps of the same channel dimension is compressed along the spatial dimension using a global average pooling module to obtain multiple scale-compressed vectors. Specifically, taking F1 as an example, the tensor dimension of F1 is now... C represents the number of channels in F1. as well as Let F1 represent the height and width, respectively. F1 is compressed along the spatial dimensions (height and width) using the first global average pooling module, compressing the spatial information of each channel into a single value, thus changing the tensor dimension to... This yields the scale-compressed vector. Similarly, F2 and F3 are compressed into C×1×1 vectors via the global average pooling module, resulting in three scale-compressed vectors.

[0035] Multiple scale-compressed vectors are concatenated along the height direction by a feature fusion module to obtain a fused vector. Specifically, three scale-compressed vectors are concatenated along the height direction to obtain the fused vector. Its tensor dimension is .

[0036] The fused vector is numerically compressed using a global average pooling module and a non-linear activation module to obtain activation values. Specifically, the fused vector... The fused vector is compressed to a value between 0 and 1 through a second global average pooling module and the Sigmoid activation function in the non-linear activation module, serving as the activation value for the gated unit. The Sigmoid activation function is a non-linear activation function that maps input values ​​to output values ​​between 0 and 1; in specific implementations, it is used to compress the fused vector... Nonlinear compression is performed to obtain the activation value of the gating unit. The activation value is a value calculated by the gating unit based on the predicted feature map. Its value ranges from 0 to 1 and is used to characterize the viewpoint category of the currently input image data to be tested, and to select the corresponding target prediction head accordingly.

[0037] The gating unit determines the network structure of the prediction branch based on the activation value. In a specific implementation, when the activation value is less than or equal to 0.5, the first prediction head is selected to output the two-dimensional bounding box prediction result; when the activation value is greater than 0.5, the second prediction head is selected to output the key point prediction result.

[0038] The beneficial effects of this step are as follows: by using gating units to jointly calculate activation values ​​from multiple prediction feature maps of different scales, and using these activation values ​​to realize a variable prediction head network structure, the object detection model can adaptively select the appropriate prediction head for output based on different camera perspectives, thus exhibiting strong generalization performance. At the same time, through the cascaded processing of 1×1 convolution, global average pooling, feature fusion, and nonlinear activation, multiple prediction feature maps of different scales are effectively compressed into a single activation value, resulting in high computational efficiency and easy training convergence.

[0039] S6: Select a target prediction head from multiple candidate prediction heads based on the activation value. The specific implementation is as follows: In a preferred embodiment, the plurality of candidate prediction heads includes a first prediction head and a second prediction head.

[0040] The first prediction head is used to output the 2D bounding box prediction results. The first prediction head refers to the network structure part of the object detection model that outputs the 2D bounding box prediction results. The 2D bounding box prediction result is an object detection output format that uses a rectangular box parallel to the horizontal and vertical axes of the pixel coordinate system to identify the position of the target object in the image. It is uniquely determined by four parameters: the x-coordinate of the center point, the y-coordinate of the center point, the width, and the height. The 2D bounding box prediction result includes the x-coordinate of the center point, the y-coordinate of the center point (i.e., the center point coordinates), the width, the height, and the foreground probability. The foreground probability is the probability value used in the object detection model to determine whether the target is foreground or background. Its value is between 0 and 1; the closer the value is to 1, the greater the probability that the region contains the target object; the closer the value is to 0, the greater the probability that the region does not contain the target object and is background. The first prediction head is specifically a 1×1 convolutional module, and the number of output channels of this convolutional module is... ,in, The number represents the number of target categories. 5 represents the x-coordinate, y-coordinate, width, height, and foreground probability of the target object's center point. 3 represents three different anchor box scales. Anchor boxes are a predefined set of reference boxes with different sizes and aspect ratios used in the target detection algorithm to generate candidate regions at each location in the feature map. The model regresses the target position by predicting the offset of the anchor box relative to the ground truth bounding box. In this specific implementation, three different anchor box scales are used. In this implementation, the first prediction head is used to output two-dimensional bounding box prediction results for a standard viewpoint. A standard viewpoint refers to a shooting angle between 30 and 75 degrees between the camera's shooting angle and the ground. Under this viewpoint, the visible area of ​​the target vehicle includes the front or rear of the vehicle, the sides, and the roof.

[0041] The second prediction head is used to output keypoint prediction results. The second prediction head refers to the network structure part of the object detection model that outputs keypoint prediction results. Keypoint prediction results include the coordinates of multiple keypoints of the target object and foreground probabilities. Keypoint prediction results refer to the pixel coordinates of specific locations on the target object with clear semantic meaning in the image. In a specific implementation, this refers to the four corner positions of the vehicle: the left front headlight, left rear taillight, right rear taillight, and right front headlight. Multiple keypoints include the keypoints of the target vehicle's left front headlight, left rear taillight, right rear taillight, and right front headlight. The keypoint prediction results include the coordinates of multiple keypoints of the target object and foreground probabilities. The second prediction head is specifically a 1×1 convolutional module, and the number of output channels of this convolutional module is... ,in, The number of target categories is represented by 9, indicating the keypoint coordinates (including x and y coordinates) of the left front headlight, left rear taillight, right rear taillight, and right front headlight, as well as the foreground probability. 3 represents three different anchor frame scales. In a specific implementation, the second prediction head is used to output keypoint prediction results for a top-down view. A top-down view refers to a shooting angle where the camera's angle relative to the ground is greater than 75 degrees; from this angle, only the roof area of ​​the target vehicle is visible.

[0042] The target prediction head is selected as follows: when the activation value of the gating unit is less than or equal to 0.5, the first prediction head is selected as the target prediction head from multiple candidate prediction heads; when the activation value of the gating unit is greater than 0.5, the second prediction head is selected as the target prediction head from multiple candidate prediction heads.

[0043] When there are three or more candidate predictor heads, the target predictor head is selected as follows: the range of activation values ​​is divided into multiple consecutive numerical intervals, with each interval corresponding to a candidate predictor head. The numerical interval to which the activation value belongs is determined, and the candidate predictor head corresponding to that interval is selected as the target predictor head. For a specific example with three candidate predictor heads: when the activation value is between 0 and 0.33, the first predictor head is selected as the target predictor head; when the activation value is between 0.33 and 0.67, the second predictor head is selected as the target predictor head; when the activation value is between 0.67 and 1, the third predictor head is selected as the target predictor head.

[0044] The beneficial effect of this step is that by adaptively selecting the target prediction head through the activation value of the gating unit, a single model can flexibly switch the output format according to the viewpoint characteristics of the input image data to be tested, without the need for manual intervention or additional maintenance of multiple independent models, which significantly reduces deployment and maintenance costs.

[0045] S7 inputs prediction feature maps of various scales into the target prediction head to obtain the target detection results corresponding to the image data to be tested. The specific implementation method is as follows: The three different scale prediction feature maps F1, F2, and F3 output from the neck network in S4 are input into the target prediction head selected in S6. If the activation value is less than or equal to 0.5, the target prediction head is the first prediction head. The first prediction head performs pointwise convolution on the three prediction feature maps and outputs the 2D bounding box prediction result at each location, including the center point coordinates, width, height, and foreground probability of the target object. If the activation value is greater than 0.5, the target prediction head is the second prediction head. The second prediction head performs pointwise convolution on the three prediction feature maps and outputs the keypoint prediction result at each location, including the coordinates of multiple keypoints of the target object and the foreground probability. The multiple keypoints include the left front headlight keypoint, left rear taillight keypoint, right rear taillight keypoint, and right front headlight keypoint of the target vehicle. The final 2D bounding box prediction result or keypoint prediction result is the target detection result corresponding to the image data to be tested. The target detection result refers to the final result output by the target detection model after detecting the input image data to be tested. In specific implementations, it is the two-dimensional bounding box prediction result or the key point prediction result, which depends on the target prediction head selected by the gating unit activation value.

[0046] The object detection results will be used for downstream task logic judgment. Downstream tasks refer to the application logic that needs to be executed after the object detection results are output. In a specific implementation, this is the determination of parking space occupancy status. It is also applicable to other application scenarios such as vehicle trajectory tracking, vehicle counting, and illegal parking detection. For example, in the scenario of a highway service area parking lot, the 2D bounding box prediction results or key point prediction results can be used to determine whether a parking space is occupied by a vehicle.

[0047] The beneficial effects of this step are as follows: the target prediction head performs the final prediction on the multi-scale prediction feature map, outputs a precisely fitting two-dimensional bounding box for the conventional view, and outputs the key point coordinates of the four corner points of the vehicle for the top-down view. It achieves efficient extraction of target pixels under different camera views, reduces background redundancy information in the detection results, and improves positioning accuracy.

[0048] In addition, it also includes the training phase of the object detection model, and the training process of the object detection model is described below.

[0049] In a preferred embodiment, the training process for the object detection model is as follows: Training image data labeled with different viewpoint classification tags are input into the object detection model. The gating unit activation value loss function is used for parameter gradient backpropagation and parameter convergence of the object detection model. The specific implementation is as follows: Training image data for highway service area parking lot scenes was collected. The training image data refers to the image dataset used to train the object detection model, including images from both regular and overhead perspectives. Each image is labeled with a viewpoint classification label and corresponding detection annotation information, i.e., a 2D bounding box or keypoint coordinates. The training image data was acquired using a high-pole camera, with the camera installed at a height of no less than 6 meters and an installation angle between 30 and 90 degrees. The shooting scene should include rich lighting and weather conditions, such as daytime, nighttime, sunny days, and rainy / snowy days. The vehicle types in the training image data should include cars, trucks, and buses to improve the diversity of the training data (consistent with the acquisition conditions of the image data to be tested).

[0050] The training dataset is divided based on the camera shooting angle. Data with an angle between 30 and 75 degrees from the ground is defined as the regular viewpoint dataset and assigned a regular viewpoint classification label. Data with an angle greater than 75 degrees from the ground is defined as the overhead viewpoint dataset and assigned an overhead viewpoint classification label. In a specific implementation, the regular viewpoint classification label is represented by "0", and the overhead viewpoint classification label is represented by "1".

[0051] For the standard view dataset, a 2D bounding box annotation tool is used to annotate the target objects (i.e., the target vehicles). The labels include the category index, the x-coordinate and y-coordinate of the center point, the width, the height, and the standard view classification label. The category index indicates the vehicle model category, the x-coordinate and y-coordinate of the center point represent the pixel coordinates of the center point of the 2D bounding box, and the width and height represent the width and height of the 2D bounding box. For the overhead view dataset, a keypoint annotation tool is used to annotate the vehicle's keypoints. The labels include the category index, the x-coordinate and y-coordinate of the left front headlight keypoint, the x-coordinate and y-coordinate of the left rear taillight keypoint, the x-coordinate and y-coordinate of the right rear taillight keypoint, the x-coordinate and y-coordinate of the right rear taillight keypoint, the x-coordinate and y-coordinate of the right front headlight keypoint, and the overhead view classification label. The sequential indices of the four keypoints—left front headlight, left rear taillight, right rear taillight, and right front headlight—are 0, 1, 2, and 3, respectively, used for calculating the keypoint loss function.

[0052] The labeled training image data is input into the object detection model for training. Compared with traditional object detection training tasks, due to the introduction of gating units, a gating unit activation value loss function is needed during training to achieve gradient backpropagation and convergence of relevant module parameters. The gating unit activation value loss function is a loss function used to measure the difference between the activation value output by the gating unit and the true classification label of the training image data. Specifically, it is in the form of a cross-entropy loss function, and gradient backpropagation and convergence of relevant gating unit parameters are achieved through backpropagation. The predicted value of the gating unit activation value loss function is the activation value output by the gating unit, that is, the non-linear result obtained by the Sigmoid activation function, whose value is between 0 and 1. The true value is the classification label labeled in the training image data, that is, the classification label "0" for conventional viewpoint or the classification label "1" for top-down viewpoint. The gating unit activation value loss function is calculated based on the classification label of each training image data and the corresponding activation value of each training image data. Its calculation formula is as follows: ; in, The loss value corresponding to the activation value loss function of the gated unit. For the first Classification labels for the training image data. For the first The activation values ​​corresponding to the training image data. This represents the number of images in the current batch. Parameter gradient backpropagation refers to the process during deep learning training where the gradient of the loss function with respect to the parameters of each network layer is calculated using the chain rule, and this gradient is propagated layer by layer from the output layer to the input layer to update the network parameters. Parameter convergence refers to the state during deep learning training where, after multiple iterations, the network parameters reach a stable value where the loss function value stabilizes and the parameter changes approach zero, indicating that the model has been successfully trained. During training iterations, the gradient of the loss function is propagated layer by layer using the backpropagation algorithm, updating all trainable parameters in the shared backbone network, neck network, gating units, and prediction head of the object detection model, gradually converging the model parameters to their optimal state.

[0053] Different perspectives include a first perspective and a second perspective. In a specific implementation, the first perspective is a conventional perspective, that is, the angle between the camera shooting angle and the ground is between 30 degrees and 75 degrees, which is represented by the classification label "0"; the second perspective is a top-down perspective, that is, the angle between the camera shooting angle and the ground is greater than 75 degrees, which is represented by the classification label "1".

[0054] In a preferred implementation, the total loss function of the object detection model is divided into a first-view total loss function (i.e., the conventional view total loss function) and a second-view total loss function (i.e., the overhead view total loss function). The total loss function refers to the loss function used during the training of the object detection model to comprehensively evaluate the overall difference between the model output and the labeled ground truth, and is obtained by summing multiple sub-loss functions. In specific implementations, the total loss function takes two forms: the conventional view total loss function and the overhead view total loss function.

[0055] The first-person perspective total loss function (i.e., the conventional perspective total loss function) includes the target foreground or background probability loss, target classification loss, 2D bounding box regression loss, and the loss value corresponding to the gating unit activation value loss function. The calculation formula for the first-person perspective total loss function (i.e., the conventional perspective total loss function) is as follows: ; in, For the total loss function from a conventional perspective, This is the target foreground or background probability loss, used to determine the probability that the target is either foreground or background. Classification loss for the target, For 2D bounding box regression loss, The loss value corresponds to the gating unit activation value loss function. The target foreground or background probability loss is a loss function used to supervise the model in determining whether a candidate region contains a target object, enabling the target detection model to accurately distinguish between foreground targets and background regions. The target classification loss is a loss function used to supervise the model in correctly classifying the target object category, enabling the target detection model to identify the category to which the target belongs (e.g., car, truck, bus, etc.). The 2D bounding box regression loss is a loss function used to supervise the deviation between the predicted 2D bounding box position and the labeled true bounding box position, typically using forms such as smoothed L1 loss or intersection-over-union loss. The loss value corresponding to the gating unit activation value loss function is calculated from the gating unit activation value loss function.

[0056] The second-view total loss function (i.e., the overhead view total loss function) includes the target foreground or background probability loss, target classification loss, keypoint loss, and the loss value corresponding to the gating unit activation value loss function. The calculation formula for the second-view total loss function (i.e., the overhead view total loss function) is as follows: ; in, Let the total loss function be the overhead viewpoint. Foreground or background probability loss, Classification loss for the target For key point losses, This represents the loss value corresponding to the gating unit activation value loss function. The target foreground or background probability loss and target classification loss are the same as their corresponding losses in the first-view total loss function. The keypoint loss is a loss function used to supervise the deviation between the predicted keypoint coordinates and the labeled true keypoint coordinates. In a specific implementation, it is used to supervise the prediction accuracy of four keypoints: the left front headlight, the left rear taillight, the right rear taillight, and the right front headlight. The loss value corresponding to the gating unit activation value loss function is calculated from the gating unit activation value loss function and is the same as the loss value corresponding to the gating unit activation value loss function in the first-view total loss function.

[0057] After multiple rounds of iterative training, the parameters of the object detection model gradually converged, completing the model training process.

[0058] The beneficial effects of this step are as follows: by introducing the gating unit activation value loss function, the gating unit can learn the correct viewpoint classification ability during training, ensuring that the activation value output by the gating unit can accurately reflect the viewpoint category of the input image, thereby correctly selecting the corresponding prediction head in the inference stage; at the same time, by designing total loss functions containing different regression loss terms for conventional viewpoints and overhead viewpoints respectively, the model can optimize the corresponding prediction head parameters for different viewpoints, thereby improving the detection accuracy under each viewpoint.

[0059] The trained object detection model is deployed to edge computing devices, such as embedded AI computing platforms, to achieve real-time vehicle target detection in highway service area parking lot scenarios. When the camera shooting angle corresponding to the viewpoint of the image data transmitted by the business platform is between 30 and 75 degrees, the activation value of the gating unit is less than or equal to 0.5, and the object detection model selects the first prediction head to output the 2D bounding box detection result. When the camera shooting angle of the image viewpoint is greater than 75 degrees, the vehicle target is only visible from the top, the activation value of the gating unit is greater than 0.5, and the object detection model selects the second prediction head to output the keypoint detection result. The output results will be used for downstream task logic judgment, such as determining the parking space occupancy status.

[0060] The beneficial effects of the target detection method based on a shared backbone network and gating units provided by this invention are as follows: Multi-scale feature extraction and fusion are achieved by sharing the backbone and neck networks. The gating unit calculates the activation value based on the multi-scale predicted feature map and selects a target prediction head from multiple candidate prediction heads to output the detection result. This allows a single target detection model to adaptively select the appropriate prediction head for output based on different viewpoints of the input image, eliminating the need to maintain multiple independent network models for different viewpoints, thus reducing data maintenance and development costs. At the same time, since different prediction heads share the same weight parameters of the backbone and neck networks, computational overhead is significantly reduced. By selecting a prediction head adapted to the viewpoint to output detection results in a specific format, background redundancy information in the detection results can be reduced, improving localization accuracy.

[0061] In the above embodiments, although the steps are numbered S1, S2, etc., they are only specific embodiments given by the present invention. Those skilled in the art can adjust the execution order of S1, S2, etc. according to the actual situation, and these situations are also within the protection scope of the present invention. It can be understood that in some embodiments, some or all of the above embodiments may be included.

[0062] Furthermore, the acquisition process of the data involved in this application follows the principles of legality, legitimacy, and necessity. Based on obtaining the explicit authorization and consent of the user, only the minimum necessary information required to achieve the purpose is collected, and data security protection obligations are fulfilled in accordance with the law.

[0063] The present invention also provides a target detection system based on a shared backbone network and gating units, the specific technical solution of which is as follows: an acquisition module, an input module, an extraction module, a fusion module, a calculation module, a selection module, and an output module; The acquisition module is used to acquire the image data to be tested, which is obtained by the image acquisition device from the target area. The input module is used to input the image data to be tested into the target detection model, which includes a shared backbone network, a neck network, gating units, and multiple candidate prediction heads. The extraction module is used to extract features from the image data to be tested through a shared backbone network to obtain an initial feature map; The fusion module is used to perform multi-scale feature fusion on the initial feature map through the neck network to obtain prediction feature maps of various different scales; The calculation module is used to determine the activation value of the image data to be tested based on the predicted feature maps of various scales through the gating unit; The selection module is used to select a target prediction head from multiple candidate prediction heads based on the activation value; The output module is used to input prediction feature maps of various scales into the target prediction head to obtain the target detection results corresponding to the image data to be tested.

[0064] It should be noted that the beneficial effects of the target detection system based on a shared backbone network and gating units provided in the above embodiments are the same as those of the target detection method based on a shared backbone network and gating units described above, and will not be repeated here. Furthermore, the system provided in the above embodiments is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the system can be divided into different functional modules according to the actual situation to complete all or part of the functions described above. In addition, the system and method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process is detailed in the method embodiments, and will not be repeated here.

[0065] like Figure 4 As shown, an embodiment of the present invention provides a computer device 300, which includes a processor 320 coupled to a memory 310. The memory 310 stores at least one computer program 330, which is loaded and executed by the processor 320 to enable the computer device 300 to implement any of the above-described methods. Specifically: The computer device 300 can vary considerably due to differences in configuration or performance. It may include one or more processors 320 (Central Processing Units, CPUs) and one or more memories 310. The one or more memories 310 store at least one computer program 330, which is loaded and executed by the one or more processors 320 to enable the computer device 300 to implement the target detection method based on a shared backbone network and gating units provided in the above embodiments. Of course, the computer device 300 may also have wired or wireless network interfaces, a keyboard, and input / output interfaces for input and output. The computer device 300 may also include other components for implementing device functions, which will not be elaborated here.

[0066] An embodiment of the present invention provides a computer-readable storage medium storing at least one computer program, which is loaded and executed by a processor to enable a computer to implement any of the above-described methods.

[0067] Alternatively, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, a floppy disk, and an optical data storage device, etc.

[0068] In an exemplary embodiment, a computer program product or computer program is also provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform any of the target detection methods based on a shared backbone network and gating units described above.

[0069] It should be noted that the terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and do not imply a specific order or sequence. Where appropriate, the order of use of similar objects can be interchanged so that the embodiments of this application described herein can be implemented in an order other than that shown or described.

[0070] Those skilled in the art will recognize that this invention can be implemented as a system, method, or computer program product. Therefore, this disclosure can be specifically implemented in the following forms: it can be entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software, generally referred to herein as a "circuit," "module," or "system." Furthermore, in some embodiments, the invention can also be implemented as a computer program product contained in one or more computer-readable media, which includes computer-readable program code.

[0071] Any combination of one or more computer-readable media may be used. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device.

[0072] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A target detection method based on a shared backbone network and gating units, characterized in that, include: S1, acquire the image data to be tested, which is obtained by an image acquisition device capturing images of the target area; S2, the image data to be tested is input into the target detection model, the target detection model includes a shared backbone network, a neck network, a gating unit and multiple candidate prediction heads; S3, feature extraction is performed on the image data to be tested through the shared backbone network to obtain an initial feature map; S4, Multi-scale feature fusion is performed on the initial feature map through the neck network to obtain prediction feature maps of various different scales; S5, the gating unit determines the activation value of the image data to be tested based on the predicted feature maps of various different scales; S6, Select a target prediction head from the plurality of candidate prediction heads according to the activation value; S7, input the prediction feature maps of various different scales into the target prediction head to obtain the target detection result corresponding to the image data to be tested.

2. The target detection method based on a shared backbone network and gating units according to claim 1, characterized in that, The gating unit includes a 1×1 convolution module, a global average pooling module, a feature fusion module, and a non-linear activation module; Determining the activation value of the image data to be tested specifically includes: The number of channels in the predicted feature map at each scale is mapped to the same dimension via the 1×1 convolution module to obtain a multi-scale feature map with the same channel dimension. Each scale feature map of the multi-scale feature map with the same channel dimension is compressed along the spatial dimension by the global average pooling module to obtain multiple scale compression vectors. The multiple scale-compressed vectors are concatenated along the height direction by the feature fusion module to obtain a fused vector; The fusion vector is numerically compressed using the global average pooling module and the nonlinear activation module to obtain the activation value.

3. The target detection method based on a shared backbone network and gating units according to claim 1, characterized in that, The plurality of candidate prediction heads includes a first prediction head and a second prediction head; The first prediction head is used to output a two-dimensional bounding box prediction result, which includes the center point coordinates, width, height, and foreground probability of the target object. The second prediction head is used to output keypoint prediction results, which include the coordinates of multiple keypoints of the target object and the foreground probability.

4. The target detection method based on a shared backbone network and gating units according to claim 1, characterized in that, It also includes training the target detection model: inputting training image data labeled with different viewpoint classification labels into the target detection model, and using the gated unit activation value loss function to perform parameter gradient backpropagation and parameter convergence of the target detection model; The gating unit activation value loss function is calculated based on the classification label of each training image data and the activation value corresponding to each training image data; the different perspectives include a first perspective and a second perspective.

5. The target detection method based on a shared backbone network and gating units according to claim 4, characterized in that, The total loss function of the target detection model is divided into a first-view total loss function and a second-view total loss function. The first viewpoint total loss function includes target foreground or background probability loss, target classification loss, two-dimensional bounding box regression loss, and the loss value corresponding to the gating unit activation value loss function; The second perspective total loss function includes target foreground or background probability loss, target classification loss, keypoint loss, and the loss value corresponding to the gating unit activation value loss function.

6. A target detection system based on a shared backbone network and gating units, characterized in that, include: The module includes an acquisition module, an input module, an extraction module, a fusion module, a calculation module, a selection module, and an output module. The acquisition module is used to acquire the image data to be tested, which is obtained by the image acquisition device from the target area. The input module is used to input the image data to be tested into the target detection model, which includes a shared backbone network, a neck network, a gating unit, and multiple candidate prediction heads. The extraction module is used to extract features from the image data to be tested through the shared backbone network to obtain an initial feature map. The fusion module is used to perform multi-scale feature fusion on the initial feature map through the neck network to obtain prediction feature maps of various different scales; The calculation module is used to determine the activation value of the image data to be tested based on the predicted feature maps of various different scales through the gating unit; The selection module is used to select a target prediction head from the plurality of candidate prediction heads according to the activation value; The output module is used to input the prediction feature maps of various scales into the target prediction head to obtain the target detection result corresponding to the image data to be tested.

7. A target detection system based on a shared backbone network and gating units according to claim 6, characterized in that, The gating unit includes a 1×1 convolution module, a global average pooling module, a feature fusion module, and a non-linear activation module; Determining the activation value of the image data to be tested specifically includes: The number of channels in the predicted feature map at each scale is mapped to the same dimension via the 1×1 convolution module to obtain a multi-scale feature map with the same channel dimension. Each scale feature map of the multi-scale feature map with the same channel dimension is compressed along the spatial dimension by the global average pooling module to obtain multiple scale compression vectors. The multiple scale-compressed vectors are concatenated along the height direction by the feature fusion module to obtain a fused vector; The fusion vector is numerically compressed using the global average pooling module and the nonlinear activation module to obtain the activation value.

8. A target detection system based on a shared backbone network and gating units according to claim 6, characterized in that, The plurality of candidate prediction heads includes a first prediction head and a second prediction head; The first prediction head is used to output a two-dimensional bounding box prediction result, which includes the center point coordinates, width, height, and foreground probability of the target object. The second prediction head is used to output keypoint prediction results, which include the coordinates of multiple keypoints of the target object and the foreground probability.

9. A computer device, characterized in that, The computer device includes a processor coupled to a memory storing at least one computer program, which is loaded and executed by the processor to enable the computer device to implement a target detection method based on a shared backbone network and gating units as described in any one of claims 1 to 5.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one computer program, which is loaded and executed by a processor to enable the computer to implement a target detection method based on a shared backbone network and gating units as described in any one of claims 1 to 5.

Images