A multi-feature extraction method, device, equipment and storage medium

Abstract

Embodiments of the present application provide a multi-feature extraction method, device and equipment and storage medium, the method comprising: generating an activation point set according to a predefined activation point generation formula; constructing an activation function corresponding to an input-output mapping according to the activation point set; constructing a multi-feature extraction network module comprising at least two branches according to the activation function; and constructing a neural network recognition model according to the multi-feature extraction network module to realize multi-feature extraction of to-be-detected information based on the neural network recognition model. By using the method, the activation function is constructed according to the activation point generation formula to replace the function of the existing nonlinear activation function, thereby effectively improving the inference speed and accuracy and saving the computing resource occupation. Meanwhile, different activation functions are set as the activation function mapping rules of the two branches in the multi-feature extraction network module, which can better extract multi-scale features in the input feature tensor, thereby improving the recognition rate and recognition efficiency of the final model.

Description

Technical Field

[0001] This invention relates to the field of neural network technology, and in particular to a method, apparatus, device, and storage medium for multi-feature extraction. Background Technology

[0002] With the development of neural network technology and signal acquisition technology, deep learning-based visual perception and recognition technology is increasingly being applied in production and daily life. However, limited by hardware computing power and engineering application environments, the demand for neural network development based on multi-task integration and operator optimization is growing rapidly. The mainstream intelligent development process in industry today involves designing network structures to predict target location and category while simultaneously recognizing attribute information in other dimensions. This is followed by the engineered deployment of the network model to the corresponding hardware platform and finally integration into the product. Among these challenges, designing a good multi-task prediction structure and quickly and stably deploying the network model to the appropriate hardware platform are the main challenges in current technology development.

[0003] When training neural networks, the commonly used activation functions Mish and Swish in academia can well meet the gradient backpropagation requirements. Their function shapes are quite similar, and both satisfy the characteristics of being unbounded at the upper limit but bounded at the lower limit, smooth, and non-monotonic. Overall, their performance is superior to that of the Rectified Linear Array (ReLU) function. However, in practical deployments, their complex computation methods are not conducive to numerical calculations after network feature quantization, and their internal function operators may not be supported in some deployment environments.

[0004] In addition, in the field of object detection, in order to improve the detection performance of small targets and adjacent targets, the academic community often uses feature pyramid structures such as feature pyramid networks (FPN) to achieve multi-dimensional feature extraction. However, the existence of the pyramid structure often means an increase in network convolutional layers, which affects the final target recognition efficiency. Summary of the Invention

[0005] This invention provides a multi-feature extraction method, apparatus, device, and storage medium, which improves the recognition rate and efficiency of the final model.

[0006] In a first aspect, embodiments of the present invention provide a multi-feature extraction method, including:

[0007] Construct an activation point set based on a predefined activation point generation formula;

[0008] Based on the set of activation points, construct the activation function corresponding to the input-output mapping;

[0009] Based on the activation function, construct a multi-feature extraction network module containing at least two branches;

[0010] Based on the multi-feature extraction network module, a neural network recognition model is constructed to achieve multi-feature extraction of the information to be detected based on the neural network recognition model.

[0011] Secondly, embodiments of the present invention provide a multi-feature extraction device, comprising:

[0012] The point set determination module is used to construct an activation point set according to a predefined activation point generation formula;

[0013] The function construction module is used to construct the activation function corresponding to the input-output mapping based on the set of activation points;

[0014] The module determination module is used to construct a multi-feature extraction network module containing at least two branches based on the activation function;

[0015] The feature extraction module is used to construct a neural network recognition model based on the multi-feature extraction network module, so as to realize multi-feature extraction of the information to be detected based on the neural network recognition model.

[0016] Thirdly, embodiments of the present invention also provide an electronic device, comprising:

[0017] At least one processor; and

[0018] A memory communicatively connected to the at least one processor; wherein,

[0019] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the multi-feature extraction method as described in the first aspect embodiment.

[0020] Fourthly, embodiments of the present invention also provide a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the multi-feature extraction method as described in the first aspect embodiment.

[0021] This invention provides a multi-feature extraction method, apparatus, device, and storage medium. The method first constructs an activation point set based on a predefined activation point generation formula; then, based on the activation point set, it constructs an activation function corresponding to the input-output mapping; based on the activation function, it constructs a multi-feature extraction network module containing at least two branches; and finally, based on the multi-feature extraction network module, it constructs a neural network recognition model to achieve multi-feature extraction of the information to be detected based on the neural network recognition model. This technical solution, by constructing an activation function based on the activation point generation formula, replaces the nonlinear activation function function in the prior art, effectively improving the model's inference speed and accuracy, saving computational resources, and facilitating the engineering deployment of multi-feature extraction network modules containing activation functions. Furthermore, setting different activation functions as the activation function mapping rules for the two branches in the multi-feature extraction network module can better extract multi-scale features from the input feature tensor, thereby improving the recognition rate and efficiency of the final model.

[0022] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a flowchart illustrating a multi-feature extraction method provided in Embodiment 1 of the present invention;

[0025] Figure 2 This is a flowchart illustrating another multi-feature extraction method provided in Embodiment 2 of the present invention;

[0026] Figure 3 This is a graph showing the curve effects of different activation functions and existing activation functions in a multi-feature extraction method provided in Embodiment 2 of the present invention.

[0027] Figure 4 Example diagram of feature extraction in existing technology;

[0028] Figure 5 This is an example diagram illustrating feature extraction in a multi-feature extraction method provided in Embodiment 2 of the present invention;

[0029] Figure 6 This is an example of the effect of multi-feature extraction on the information to be detected in existing technologies;

[0030] Figure 7 This is an example diagram illustrating the effect of a multi-feature extraction method provided in Embodiment 2 of the present invention on target detection of the information to be detected;

[0031] Figure 8 This is a flowchart illustrating the execution of a multi-feature extraction method in a specific application scenario, as provided in Embodiment 2 of the present invention.

[0032] Figure 9 This is a schematic diagram of the structure of a multi-feature extraction device provided in Embodiment 3 of the present invention;

[0033] Figure 10 This is a schematic diagram of the structure of an electronic device provided in Embodiment 4 of the present invention. Detailed Implementation

[0034] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0035] It should be noted that the terms "original," "target," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0036] In existing technologies, when training neural networks, the commonly used Mish and Swish activation functions in academia can well meet the gradient backpropagation requirements. Both functions have similar shape distributions and satisfy the characteristics of being unbounded at the upper limit but with a lower limit, smooth, and non-monotonic, generally outperforming the ReLU function. However, in practical deployments, their complex computation methods are not conducive to numerical calculations after network feature quantization, and their internal function operators may not be supported in some deployment environments.

[0037] In the field of object detection, to improve the detection performance of small and adjacent targets, the academic community commonly uses feature pyramid structures like FPN to extract features in multiple dimensions. However, the existence of a pyramid structure often means an increase in the number of convolutional layers in the network, affecting the final recognition efficiency. Furthermore, small targets are prone to being missed during object detection. For example, in vehicle-mounted object detection, the varying distances of targets from the vehicle sensor result in target regions of varying sizes in the imaging signal. Larger targets are easily detected by the network, while smaller targets, due to their smaller feature regions, become difficult to detect as the receptive field increases with the number of layers in the neural network. In forward-looking vehicle target detection, traffic lights, being small targets themselves, combined with the long imaging distance, easily lead to missed detections. Therefore, a method is needed to address these issues.

[0038] Example 1

[0039] Figure 1 This is a flowchart illustrating a multi-feature extraction method provided in Embodiment 1 of the present invention. This method is applicable to situations requiring multi-feature extraction of information to be detected. The method can be executed by a multi-feature extraction device, which can be implemented in hardware and / or software and can be configured in an electronic device. For example... Figure 1 As shown, the multi-feature extraction method provided in this embodiment can specifically include the following steps:

[0040] S101. Construct an activation point set according to a predefined activation point generation formula.

[0041] Considering the distribution characteristics of the Mish and Swish activation functions in the existing technology, Mish and Swish have similar distribution shapes in the (-∞, 0) interval. When the input value of the horizontal axis is very small, its gradient value changes little, while in a range close to 0, there is nonlinearity and the gradient distribution can be both positive and negative. This embodiment constructs an activation function based on the activation point generation formula to replace the nonlinear and complex activation function in the existing technology.

[0042] Specifically, the activation point generation formula can be understood as the formula used to determine each activation point in the curve corresponding to the activation function. In this embodiment, a linear function passing through the origin and the boundary values ​​of the segment are used to determine the position of each activation point, and then the mapping of the linear activation function of the segment interval is realized through two adjacent activation points.

[0043] It should be noted that in this embodiment, the activation function corresponding to the input-output mapping is constructed by setting the distribution of activation points within a defined interval. In this embodiment, the corresponding activation point positions can be determined according to a pre-generated activation point generation formula. Specifically, determining the corresponding activation point positions according to the activation point generation formula involves substituting the x-coordinate of the activation point into the activation point generation formula to obtain the corresponding y-coordinate. Based on the x-coordinate and the corresponding y-coordinate, the positions of each activation point are determined.

[0044] The activation point generation formula is determined by the initial activation point formula, which contains two hyperparameters. By selecting appropriate hyperparameter values ​​and substituting them into the initial activation point formula, the corresponding activation point generation formula can be obtained. It is clear that different activation functions can be obtained by setting different hyperparameters. In this embodiment, it is preferable to set two sets of hyperparameters to determine two activation functions as the activation functions corresponding to the input-output mapping.

[0045] S102. Based on the set of activation points, construct the activation function corresponding to the input-output mapping.

[0046] Specifically, after determining the set of activation points in the above steps, adjacent activation points are linearly fitted to form a series of line segments. This process can be understood as piecewise linear fitting, and the mapping relationship corresponding to this series of line segments is the activation function corresponding to the input-output mapping.

[0047] S103. Based on the activation function, determine the multi-feature extraction network module containing at least two branches.

[0048] The multi-feature extraction network module processes the input feature tensor to obtain a multi-feature output tensor. Compared to the computational overhead of existing encoder-decoder structures, which require additional feature pyramid network layers such as FPN and PathIntegral Based Convolution and Pooling for Graph Neural Networks (PAN), this embodiment directly outputs the data serially from the encoder network. The encoder network still uses traditional residual network spatial structures such as ResNet and lightweight deep neural network spatial structures like MobileNet, but replaces the convolutional combinations with the dual-branch structure provided in this embodiment. The convolutional combination includes Convolution (Conv) + Batch Normalization (BN) + Activation (Act). The dual-branch structure in this embodiment can be understood as two feature extraction modules, each containing a convolutional layer, normalization, and activation function. However, the two feature extraction modules use convolutional layers with different kernel sizes and different activation functions. By introducing two types of convolutional kernels with two branch structures, it is possible to extract features from the input feature tensor at different scales. At the same time, by setting different linear activation function fitting intervals, the value range of the output features can be broadened, effectively improving feature diversity, thereby achieving more efficient feature extraction and improving the final performance of the network.

[0049] After obtaining two activation functions as the corresponding activation functions for the input-output mapping in the above steps, one of the activation functions can be used as the activation function of one branch, and the other activation function can be used as the activation function of the other branch, thus forming a multi-feature extraction network module containing two branches.

[0050] S104. Based on the multi-feature extraction network module, construct a neural network recognition model to realize multi-feature extraction of the information to be detected based on the neural network recognition model.

[0051] In this context, the information to be detected can be understood as the original input information for feature extraction. This information can be an image, digital information, etc. For example, an image can be downloaded from the network, uploaded manually, or captured in real-time by an image acquisition device; no specific limitations are imposed here. The neural network recognition model can be understood as a model used to extract features from the information to be detected. In this embodiment, the main structure of the neural network recognition model remains unchanged; it can employ a residual network spatial structure or a lightweight deep neural network spatial structure. One or more layers of this multi-feature extraction network module are deployed in the neural network recognition model, enabling multi-feature extraction from the input features of this layer. The extracted multi-features are then concatenated as the output features of this layer, achieving the purpose of multi-feature extraction.

[0052] Furthermore, the neural network recognition model can be trained using labeled samples to obtain a trained neural network recognition model that meets the training accuracy requirements. Specifically, since the set layer of the neural network model consists of multi-feature extraction network modules, after the information to be detected is input into the neural network recognition model, the input features are obtained before branch processing when passing through the set layer containing the multi-feature extraction network modules. The multi-feature extraction network modules process the input features, inputting them into two branches for feature processing. Each branch can be understood as a feature extraction module, and each feature extraction module includes a convolutional layer, normalization, and activation function. The convolutional kernel sizes of the convolutional layers in the two branches are different, and different activation functions are used.

[0053] Continuing from the above description, after processing the information to be detected to obtain input features, these features are further input into two branches. It can be understood that the dual-branch structure, by introducing two scales of convolutional kernels, achieves feature extraction from the input feature tensor at different scales. Simultaneously, by setting different activation function fitting intervals, it broadens the value range of the output features, effectively improving feature diversity and thus achieving more efficient feature extraction, thereby enhancing the final performance of the multi-feature extraction network module. Specifically, this multi-feature extraction network module is deployed in one or more layers within the neural network recognition model. This allows for multi-feature extraction of the input features from this layer. The input features are processed by the feature extraction modules of the two branches, obtaining corresponding output features. The output features obtained from the two branches are then concatenated to obtain the output features of this layer. Finally, the output features are processed by the network layers following the layer to obtain the final output features, which serve as the output features of the entire neural network recognition model. For example, based on the final output features, the target contained in the information to be detected can be determined. When performing target detection, the detection result can be described in the form of the bounding box containing the target. For example, the bounding box is a rectangle, which can be determined by the coordinates of the top-left and bottom-right vertices of the rectangle. It should be noted that in this embodiment, the two-branch approach is only one implementation of the multi-feature extraction network module; the multi-feature extraction network module can also be configured as a network structure containing multiple branches.

[0054] For example, assuming the information to be detected is an image to be detected, and the application scenario is to identify targets in the image, determining the targets contained in the image can be done by determining the specific location of the targets or determining the region where the targets are located. For instance, in the field of vehicle target detection, due to the varying distances of targets from the vehicle sensor, the target regions in the imaging signal vary in size. Large targets are easily detected by the network, while small targets, limited by their smaller feature regions, become difficult to detect as the receptive field increases with the number of layers in the neural network. For example, in vehicle forward-looking target detection, traffic lights are small targets, and the long imaging distance easily leads to missed detections. In this embodiment, the image of the vehicle's forward view is captured as the information to be detected. This information is processed by a neural network recognition model containing a multi-feature extraction network module. The dual-branch multi-feature extraction network module can obtain multi-scale features of the information to be detected. Based on these multi-scale features, effective detection of small-scale targets can be achieved.

[0055] This invention provides a multi-feature extraction method. The method first constructs an activation point set based on a predefined activation point generation formula; then, based on the activation point set, it constructs an activation function corresponding to the input-output mapping; based on the activation function, it constructs a multi-feature extraction network module containing at least two branches; and finally, based on the multi-feature extraction network module, it constructs a neural network recognition model to achieve multi-feature extraction of the information to be detected based on the neural network recognition model. This technical solution, by constructing an activation function based on the activation point generation formula, replaces the nonlinear activation function function in existing technologies, effectively improving the model's inference speed and accuracy, saving computational resources, and facilitating the engineering deployment of the multi-feature extraction network module containing this activation function. Furthermore, setting different activation functions as the activation function mapping rules for the two branches in the multi-feature extraction network module allows for better extraction of multi-scale features from the input feature tensor, thereby improving the recognition rate and efficiency of the final model.

[0056] Example 2

[0057] Figure 2 This is a flowchart illustrating another multi-feature extraction method provided in Embodiment 2 of the present invention. This embodiment is a further optimization of the above embodiment. In this embodiment, the "constructing an activation point set according to a pre-generated activation point generation formula" is further optimized, the "constructing an activation function corresponding to the input-output mapping according to the activation point set" is further optimized, the "determining a multi-feature extraction network module containing at least two branches according to the activation function" is further optimized, and the "constructing a neural network recognition model according to the multi-feature extraction network module to realize multi-feature extraction of the information to be detected based on the neural network recognition model" is further optimized.

[0058] like Figure 2 As shown in the figure, this embodiment 2 provides a multi-feature extraction method, which specifically includes the following steps:

[0059] S201. Input the x-coordinates of each activation point within the set interval into the activation point generation formula to obtain the output data, and use the output data as the y-coordinates of the corresponding activation points.

[0060] Specifically, the defined interval can be understood as the distribution range of activation points corresponding to the activation function. To construct the activation function corresponding to the input-output mapping, it is necessary to first determine the distribution of each activation point in the curve corresponding to the activation function. The abscissa of each activation point is input into the activation point generation formula to obtain the ordinate of the corresponding activation point. It should be noted that since the multi-feature extraction network module provided in this implementation has at least a dual-branch structure, each branch includes a different activation function. By setting different values ​​of the hyperparameters in the activation point determination formula, different activation point generation formulas can be determined. Therefore, the activation point generation formula in this step includes two activation point formulas, with one set of hyperparameters corresponding to one activation point formula. Inputting the abscissa of each activation point into the two activation point formulas respectively yields two sets of corresponding activation point ordinates.

[0061] Furthermore, the steps for generating the activation point generation formula include:

[0062] a1) Substitute the first parameter value and the second parameter value as hyperparameter values ​​into the initial activation point formula to determine the first activation point formula and the second activation point formula respectively.

[0063] The formula for the initial activation point is: Where x represents the x-coordinate of the activation point, f(x) represents the y-coordinate of the activation point, and α and β represent hyperparameters. In practical applications, the values ​​of the hyperparameters can be set independently, and the appropriate values ​​can be selected based on the experimental results. It is known that different values ​​correspond to different distributions of activation points.

[0064] In this embodiment, the initial activation point formula includes hyperparameters, the values ​​of which can be set according to actual conditions. By controlling two hyperparameters, this embodiment can introduce nonlinear mapping effects within the range where the x-coordinate is less than zero (i.e., x < 0), while maintaining a wider value range and a more uniform gradient distribution, which is beneficial for network training and inference. Replacing the complex nonlinear activation function with a piecewise linear fitting approach meets engineering deployment requirements while improving computational efficiency. Furthermore, by controlling the hyperparameters, different nonlinear distribution spline shapes can be achieved, making it suitable for training different neural networks.

[0065] The specific values ​​of the first and second hyperparameters can be determined based on the actual results. Specifically, substituting the first hyperparameter value into the initial activation point formula yields the first activation point formula; substituting the second hyperparameter value into the initial activation point formula yields the second activation point formula.

[0066] b1) Use the first activation point formula and the second activation point formula as the activation point generation formula.

[0067] Specifically, the first activation point formula and the second activation point formula are used as activation point generation formulas. Since the hyperparameter values ​​in the first activation point formula and the second activation point formula are different, the activation point distribution determined based on the first activation point formula and the second activation point formula are different. Furthermore, the curves obtained by piecewise linear fitting and the corresponding activation functions of the curves are also different.

[0068] For example, hyperparameters α=0, β=2, α=0, β=1, and α=0, β=4 are set respectively to determine the corresponding activation function in the interval (-10, 0). Figure 3 This is a graph showing the curve effects of different activation functions and existing activation functions in a multi-feature extraction method provided in Embodiment 2 of the present invention. Figure 3 As shown, different activation functions can be obtained by setting different hyperparameters. The curve of the activation function can fit the effect of the original nonlinear activation function mapping. At the same time, the desired activation function can be formed by setting the value of the hyperparameters.

[0069] S202. Based on each horizontal coordinate and its corresponding vertical coordinate, determine the set of activation points that includes each activation point.

[0070] Specifically, once the above steps determine the ordinates corresponding to the x-coordinates of each point, the positions of each activation point are essentially determined. In this embodiment, the set of activation points is denoted as the activation point set.

[0071] S203. Connect adjacent activation points to obtain line segments between adjacent activation points.

[0072] Specifically, after obtaining a series of activation points within a specified interval, connecting two adjacent endpoints of the activation point set yields line segments between adjacent activation points. Each line segment corresponds to a linear mapping function. Connecting any two adjacent activation points results in a line segment, corresponding to a linear mapping function. Within a certain interval, a series of segmented line segments can be obtained, each corresponding to a series of linear mapping functions.

[0073] S204. Use the linear mapping function corresponding to each line segment as the activation function corresponding to the input-output mapping.

[0074] Specifically, a series of piecewise linear mapping functions can ultimately fit the effect of nonlinear function mapping in existing technologies. Through piecewise linear fitting, the activation function corresponding to the input-output mapping is obtained. This activation function includes a first activation function and a second activation function. The first and second activation functions are obtained by setting different hyperparameter values.

[0075] It should be noted that in this embodiment, the activation function is determined by the distribution of activation points within a set interval, mainly referring to the portion of activation points whose x-coordinates are less than or equal to zero. For the portion of activation points whose x-coordinates are greater than zero, the corresponding activation function can be expressed as: f(x) = x, (x>0).

[0076] For example, by pre-setting the hyperparameters α=0 and β=2 as a set of values, a first activation function can be obtained; by setting α=0 and β=4 as a set of values, a second activation function can be obtained.

[0077] S205. Construct a first feature extraction module based on the first set convolutional kernel, normalization, and the first activation function.

[0078] In this embodiment, a dual-branch structure is used for feature extraction. This structure comprises two branches, or two feature extraction modules, each containing a convolutional layer, normalization, and an activation function. However, the two feature extraction modules use convolutional layers with different kernel sizes and different activation functions. By introducing two scales of convolutional kernels through the dual-branch structure, features of different scales from the input feature tensor can be extracted. Furthermore, by setting different linear activation function fitting intervals, the range of output feature values ​​can be broadened, effectively improving feature diversity and thus achieving more efficient feature extraction, ultimately enhancing the recognition rate and efficiency of the final model.

[0079] For example, the first convolutional kernel can be set to 1*1. By setting the activation points, the hyperparameters α=0 and β=2 in the formula are determined to define the distribution of activation points. Then, each adjacent activation point is piecewise fitted to obtain the first activation function. Specifically, a first feature extraction module is constructed based on the convolutional layer with the first set convolutional kernel, normalization, and the first activation function.

[0080] S206. Construct a second feature extraction module based on the convolutional layer with the second set convolutional kernel, normalization, and the first activation function.

[0081] For example, the second convolutional kernel can be set to 3*3. By setting the activation points, the hyperparameters α=0 and β=4 in the formula are determined to define the distribution of activation points. Then, adjacent activation points are piecewise fitted to obtain the second activation function. Specifically, a second feature extraction module is constructed based on the convolutional layer with the second convolutional kernel, normalization, and the second activation function.

[0082] S207. Based on the first feature extraction module and the second feature extraction module, construct a multi-feature extraction network module containing at least two branches.

[0083] Considering the computational cost of additional feature pyramid network layers such as FPN and PAN required by the existing encoder-decoder structure, this embodiment directly outputs the data serially based on the encoder network. The encoder network still uses traditional spatial structures such as ResNet and MobileNet, but replaces the convolutional combination (Conv+BN+Act) with the dual-branch structure provided in this embodiment.

[0084] For example, Figure 4 Example diagrams for feature extraction in existing technologies, such as Figure 4 As shown, after processing the information to be detected to obtain the input features, a combination of convolutional network, normalization and activation function is used for feature extraction. Specifically, the input features are processed by convolution (Conv(3*3)), normalization and activation function (Swish) to obtain the output features. Figure 5 This is an example diagram illustrating feature extraction in a multi-feature extraction method provided in Embodiment 2 of the present invention, as shown below. Figure 5 As shown, this embodiment employs a dual-branch structure, introducing convolutional kernels of 1*1 and 3*3 scales to achieve feature extraction at different scales from the input features. One branch performs convolution (Conv(1*1)), normalization, and linear fitting (with hyperparameters set to α=0, β=2, equivalent to the first activation function), while the other branch performs convolution (Conv(3*3)), normalization, and linear fitting (with hyperparameters set to α=0, β=4, equivalent to the second activation function). By setting different linear activation functions, different fitting intervals are achieved, which broadens the value range of the output features, effectively improves feature diversity, and thus achieves more efficient feature extraction, improving the final performance of the network. This dual-branch multi-scale feature extraction network structure can effectively replace the traditional single-branch scheme of convolution + normalization + activation, enhancing the network's feature extraction capabilities.

[0085] S208. Deploy the multi-feature extraction network module in the set layer of the initial network model to obtain the neural network recognition model.

[0086] The initial network model employs a residual network spatial structure or a lightweight deep neural network spatial structure. A multi-feature extraction network module can be used as a designated layer in the neural network recognition model. This designated layer can refer to one or more layers within the neural network recognition model; it can be any network layer in the neural network recognition model used as a designated layer, and a multi-feature extraction network module can be applied to this designated layer to construct the neural network recognition model.

[0087] The neural network recognition model can be trained based on a pre-labeled training sample set. For example, the pre-labeled training sample set may include a set of images whose objects are already labeled. Using this pre-labeled image training set as training samples, the initial detection model is trained, and the parameters in the initial detection model are continuously adjusted. When the detection results of the neural network recognition model meet the accuracy requirements, this trained neural network recognition model is used as the final network model for multi-feature extraction.

[0088] It should be noted that for feature extraction of the same category of information, only one neural network recognition model needs to be built. For feature extraction of different categories of information, separate neural network recognition models need to be built for each category.

[0089] S209. Obtain the initial features of the output of the previous layer in the set layer of the neural network recognition model.

[0090] Specifically, a neural network recognition model may contain a multi-layer structure. The set layer is a network layer with multiple feature extraction network modules. Before the set layer, there may be a network layer that processes the features. In this case, the information to be detected needs to be input into the network layer before the set layer in the neural network recognition model and processed sequentially to obtain the initial features output by the set layer in the preceding layer.

[0091] S210. Input the initial features into the first feature extraction module in the set layer to obtain the first output features.

[0092] Specifically, the initial features are input into the first feature extraction module in the set layer. The initial features are processed by the convolutional layer with the first set convolutional kernel, normalization, and the first activation function to obtain the processed features, which are recorded as the first output features.

[0093] S211. Input the initial features into the second feature extraction module in the set layer to obtain the second output features.

[0094] Specifically, the initial features are input into the second feature extraction module in the set layer. The input features are processed by the convolutional layer with the second set convolutional kernel, normalization, and the second activation function to obtain the processed features, which are recorded as the second output features.

[0095] S212. Perform feature concatenation processing on the first output feature and the second output feature to obtain the output feature.

[0096] Specifically, the first output feature and the second output feature are concatenated to obtain the processed feature, which is recorded as the output feature. For example, the first and second output features are concatenated in a specific order to form a new feature, which is then used as the output feature. For example, the output feature contains multi-scale features, which is more beneficial for the detection of small targets and results in more accurate target detection.

[0097] S213. Input the output features into the subsequent layer of the set layer in the neural network recognition model to obtain the final output features of the information to be detected.

[0098] Specifically, a neural network recognition model may contain multiple layers. The first layer is a network layer with multiple feature extraction network modules. After the first layer, there may be network layers that process the features. The output features need to be input into the network layers after the first layer in the neural network recognition model and processed sequentially to obtain the final output features of the entire neural network recognition model.

[0099] For example, the final output features can be represented in the form of feature vectors. For example, the final feature vectors can contain the target category and the vertex coordinates of the target bounding box. Based on this, the target contained in the detection information and the bounding box of the target can be determined.

[0100] This embodiment details the steps of constructing the activation function corresponding to the input-output mapping based on the activation point generation formula, constructing a multi-feature extraction network module based on the activation function, and extracting multiple features from the information to be detected based on the multi-feature extraction network module. Compared to the computational overhead of the traditional encoder-decoder structure requiring additional feature pyramid network layers, the above technical solution uses a dual-branch structure instead of convolution, normalization, and activation function combinations. The presence of linear activation fitting accelerates computation and eliminates the computational overhead of the decoding network, significantly improving the overall efficiency of the detection network. Furthermore, the use of dual convolution kernels and dual-domain fitting effectively enriches the information extracted from the features, thereby improving the recognition rate and efficiency of the final model.

[0101] For example, the road condition image captured by the vehicle is used as the information to be detected, and the target in the information to be detected is detected and identified. Figure 6 The image shows an example of the effect of target detection on the information to be detected in the existing technology, such as... Figure 6As shown, the information to be detected includes a car 1, a bus 2, a bicycle 3, and a traffic light 4. The information to be detected is processed by a detection model in the prior art for target detection. The detection model in the prior art includes an encoding network and a decoding network. The encoding network is a convolutional combination, i.e., convolution (Conv(3*3)), normalization, activation function (Swish), and the decoding network is an FPN network structure. It can be seen that the detection model in the prior art can identify the car 1, bus 2, and bicycle 3, but due to the small size of the traffic light 4, the traffic light 4 is not effectively identified.

[0102] Figure 7 This is an example diagram illustrating the effect of a multi-feature extraction method provided in Embodiment 2 of the present invention on target detection of the information to be detected. Figure 7 As shown, the information to be detected includes a car 1, a bus 2, a bicycle 3, and a traffic light 4. The information is processed by the multi-feature extraction network module provided in this embodiment for target detection. In this embodiment, the target is directly output serially based on the encoding network, using a dual-branch approach instead of the traditional convolution scheme during network encoding. It can be seen that the multi-feature extraction network module provided in this embodiment effectively identifies the car 1, bus 2, bicycle 3, and traffic light 4, and also effectively identifies the small target, traffic light 4. Therefore, the multi-feature extraction method provided in this embodiment can better extract multi-scale features of the input signal, thereby improving the detection effect of small-scale targets.

[0103] To more clearly illustrate the multi-feature extraction method provided in this embodiment of the invention, we will use object detection in an image as an example in a practical application scenario. For instance, Figure 8 This is a flowchart illustrating the execution of a multi-feature extraction method in a specific application scenario according to Embodiment 2 of the present invention. Figure 8 As shown, the specific steps of the multi-feature extraction method include:

[0104] S1. Substitute the first parameter value and the second parameter value as hyperparameter values ​​into the initial activation point formula to determine the first activation point formula and the second activation point formula respectively.

[0105] The formula for the initial activation point is: Where x represents the x-coordinate of the activation point, f(x) represents the y-coordinate of the activation point, and α and β represent hyperparameters.

[0106] S2. Use the first activation point formula and the second activation point formula as the activation point generation formula.

[0107] S3. Input the x-coordinates of each activation point within the set interval into the activation point generation formula to obtain the output data, and use the output data as the y-coordinates of the corresponding activation points.

[0108] S4. Based on each horizontal coordinate and its corresponding vertical coordinate, determine the set of activation points that includes each activation point.

[0109] S5. Connect adjacent activation points to obtain the line segments between each adjacent activation point.

[0110] S6. Use the linear mapping function corresponding to each line segment as the activation function corresponding to the input-output mapping.

[0111] S7. Construct a first feature extraction module based on the first set convolutional kernel, normalization, and the first activation function.

[0112] S8. Construct a second feature extraction module based on the convolutional layer with the second set convolutional kernel, normalization, and the first activation function.

[0113] S9. A multi-feature extraction network module containing at least two branches, based on the first feature extraction module and the second feature extraction module.

[0114] S10. Deploy the multi-feature extraction network module in the set layer of the initial network model to obtain the neural network recognition model.

[0115] It should be noted that for feature extraction of the same category of information, only one neural network recognition model needs to be built. For feature extraction of different categories of information, separate neural network recognition models corresponding to the thresholds need to be built.

[0116] S11. Obtain the initial features of the previous layer output in the set layer of the neural network recognition model.

[0117] S12. Input the initial features into the first feature extraction module in the set layer to obtain the first output features.

[0118] S13. Input the information to be detected into the second feature extraction module in the set layer to obtain the second output feature.

[0119] S14. Perform feature concatenation processing on the first output feature and the second output feature to obtain the output feature.

[0120] S15. Input the output features into the subsequent layer of the set layer in the neural network recognition model to obtain the final output features of the information to be detected.

[0121] Example 3

[0122] Figure 9 This is a schematic diagram of a multi-feature extraction device provided in Embodiment 3 of the present invention. This device is applicable to situations requiring multi-feature extraction of information to be detected. The multi-feature extraction device can be configured in an electronic device, such as... Figure 9As shown, the device includes: a point set determination module 31, a function construction module 32, a module determination module 33, and a feature extraction module 34; wherein,

[0123] The point set determination module 31 is used to construct an activation point set according to a predefined activation point generation formula;

[0124] The function construction module 32 is used to construct the activation function corresponding to the input-output mapping based on the set of activation points;

[0125] Module 33 is used to construct a multi-feature extraction network module containing at least two branches based on the activation function.

[0126] The feature extraction module 34 is used to construct a neural network recognition model based on the multi-feature extraction network module, so as to realize multi-feature extraction of the information to be detected based on the neural network recognition model.

[0127] This invention provides a multi-feature extraction device. The device first constructs an activation point set according to a predefined activation point generation formula; then, based on the activation point set, it constructs an activation function corresponding to the input-output mapping; based on the activation function, it constructs a multi-feature extraction network module containing at least two branches; and finally, based on the multi-feature extraction network module, it constructs a neural network recognition model to achieve multi-feature extraction of the information to be detected based on the neural network recognition model. This technical solution, by constructing an activation function according to the activation point generation formula, replaces the nonlinear activation function function in the prior art, effectively improving the model's inference speed and accuracy, saving computational resources, and facilitating the engineering deployment of multi-feature extraction network modules containing target activation functions. Furthermore, setting different activation functions as the activation function mapping rules for the two branches in the multi-feature extraction network module can better extract multi-scale features from the input feature tensor, thereby improving the recognition rate and efficiency of the final model.

[0128] Optionally, the point set determination module 31 is specifically used for:

[0129] The x-coordinates of each activation point within the set interval are input into the activation point generation formula to obtain the output data, and the output data is used as the y-coordinates of the corresponding activation points.

[0130] Based on each x-coordinate and its corresponding y-coordinate, determine the set of activation points that includes each activation point.

[0131] Optional, function building module 32, specifically used for:

[0132] Connect adjacent activation points to obtain line segments between each adjacent activation point.

[0133] The linear mapping function corresponding to each line segment is used as the activation function corresponding to the input-output mapping.

[0134] Optionally, the device also includes a formula generation module, specifically used for:

[0135] Substitute the first parameter value and the second parameter value as hyperparameter values ​​into the initial activation point formula to determine the first activation point formula and the second activation point formula respectively.

[0136] Use the formulas for the first and second activation points as the activation point generation formulas.

[0137] Optionally, the initial activation point formula is: Where x represents the x-coordinate of the activation point, f(x) represents the y-coordinate of the activation point, and α and β represent hyperparameters.

[0138] Optionally, the activation function includes a first activation function and a second activation function; module determination module 33 is specifically used for:

[0139] A first feature extraction module is constructed based on the first set convolutional kernel, normalization, and the first activation function;

[0140] The second feature extraction module is constructed based on the convolutional layer with the second set convolutional kernel, normalization, and the second activation function;

[0141] Based on the first feature extraction module and the second feature extraction module, a multi-feature extraction network module containing at least two branches is constructed.

[0142] Optionally, the feature extraction module 34 is used to construct a neural network recognition model based on the multi-feature extraction network module, including the following steps:

[0143] The multi-feature extraction network module is deployed in a set layer of the initial network model to obtain a neural network recognition model. The initial network model adopts a residual network spatial structure or a lightweight deep neural network spatial structure.

[0144] Optionally, the feature extraction module 34 is used for the step of performing multi-feature extraction on the information to be detected based on the neural network recognition model, including:

[0145] Obtain the initial features of the previous layer's output in a defined layer of a neural network recognition model;

[0146] The initial features are input into the first feature extraction module in the set layer to obtain the first output features;

[0147] The initial features are input into the second feature extraction module in the set layer to obtain the second output features;

[0148] The first output feature and the second output feature are concatenated to obtain the output feature.

[0149] The output features are input into the subsequent layers of the neural network recognition model to obtain the final output features of the information to be detected.

[0150] The multi-feature extraction device provided in the embodiments of the present invention can execute the multi-feature extraction method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the method.

[0151] Example 4

[0152] Figure 10 This is a schematic diagram of an electronic device according to Embodiment 4 of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0153] like Figure 10 As shown, the electronic device 40 includes at least one processor 41 and a memory, such as a read-only memory (ROM) 42 or a random access memory (RAM) 43, communicatively connected to the at least one processor 41. The memory stores computer programs executable by the at least one processor. The processor 41 can perform various appropriate actions and processes based on the computer program stored in the ROM 42 or loaded into the RAM 43 from storage unit 48. The RAM 43 may also store various programs and data required for the operation of the electronic device 40. The processor 41, ROM 42, and RAM 43 are interconnected via a bus 44. An input / output (I / O) interface 45 is also connected to the bus 44.

[0154] Multiple components in electronic device 40 are connected to I / O interface 45, including: input unit 46, such as keyboard, mouse, etc.; output unit 47, such as various types of monitors, speakers, etc.; storage unit 48, such as disk, optical disk, etc.; and communication unit 49, such as network card, modem, wireless transceiver, etc. Communication unit 49 allows electronic device 40 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0155] Processor 41 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 41 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 41 performs the various methods and processes described above, such as multi-feature extraction methods.

[0156] In some embodiments, the multi-feature extraction method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 48. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 40 via ROM 42 and / or communication unit 49. When the computer program is loaded into RAM 43 and executed by processor 41, one or more steps of the multi-feature extraction method described above may be performed. Alternatively, in other embodiments, processor 41 may be configured to perform the multi-feature extraction method by any other suitable means (e.g., by means of firmware).

[0157] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0158] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0159] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0160] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0161] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0162] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0163] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0164] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A target detection method, characterized in that, include: Construct an activation point set based on a predefined activation point generation formula; Based on the set of activation points, construct the activation function corresponding to the input-output mapping; Based on the activation function, construct a multi-feature extraction network module containing at least two branches; Based on the multi-feature extraction network module, a neural network recognition model is constructed to realize multi-feature extraction of the information to be detected based on the neural network recognition model. The information to be detected is a captured vehicle front view image. The neural network recognition model is trained by a labeled training sample set. The labeled training sample set specifically includes an image set in which objects contained in the image have been labeled. Based on the extracted multi-scale features, targets in the vehicle-mounted front view image are detected.

2. The method according to claim 1, characterized in that, The step of constructing an activation point set according to a predefined activation point generation formula includes: The x-coordinates of each activation point within the set interval are input into the activation point generation formula to obtain output data, and the output data is used as the y-coordinates of the corresponding activation points. Based on each of the stated horizontal coordinates and corresponding vertical coordinates, determine the set of activation points containing each of the stated activation points.

3. The method according to claim 1, characterized in that, The step of constructing the activation function corresponding to the input-output mapping based on the activation point set includes: Connect adjacent activation points in the activation point set to obtain line segments between each adjacent activation point. The linear mapping function corresponding to each line segment is used as the activation function corresponding to the input-output mapping.

4. The method according to claim 1, characterized in that, The steps for generating the activation point generation formula include: Substitute the first parameter value and the second parameter value as hyperparameter values ​​into the initial activation point formula to determine the first activation point formula and the second activation point formula respectively. The first activation point formula and the second activation point formula are used as the activation point generation formula.

5. The method according to claim 4, characterized in that, The formula for the initial activation point is: ,in, Represents the x-coordinate of the activation point. Represents the y-coordinate of the activation point. and This represents hyperparameters.

6. The method according to claim 1, characterized in that, The activation function includes a first activation function and a second activation function; The step of constructing a multi-feature extraction network module containing at least two branches based on the activation function includes: A first feature extraction module is constructed based on the first set convolutional kernel, normalization, and the first activation function; The second feature extraction module is constructed based on the convolutional layer with the second set convolutional kernel, normalization, and the second activation function; Based on the first feature extraction module and the second feature extraction module, a multi-feature extraction network module containing at least two branches is constructed.

7. The method according to claim 1, characterized in that, The step of constructing a neural network recognition model based on the multi-feature extraction network module includes: The multi-feature extraction network module is deployed in a set layer of the initial network model to obtain a neural network recognition model. The initial network model adopts a residual network spatial structure or a lightweight deep neural network spatial structure.

8. The method according to claim 7, characterized in that, The multi-feature extraction of the information to be detected based on the neural network recognition model includes: The vehicle front view image is input into the neural network recognition model to obtain the initial features of the vehicle front view image output by the designated layer in the neural network recognition model. The initial features are input into the first feature extraction module in the set layer to obtain the first output features of the vehicle front view image; The initial features are input into the second feature extraction module in the set layer to obtain the second output features of the vehicle front view image; The first output feature and the second output feature are concatenated to obtain the output features of the vehicle front view image. The output features are input into the subsequent layer of the defined layer in the neural network recognition model to obtain the final output features of the vehicle front view image.

9. A target detection device, characterized in that, include: The point set determination module is used to construct an activation point set according to a predefined activation point generation formula; The function construction module is used to construct the activation function corresponding to the input-output mapping based on the set of activation points; The module determination module is used to construct a multi-feature extraction network module containing at least two branches based on the activation function; The feature extraction module is used to construct a neural network recognition model based on the multi-feature extraction network module, so as to realize multi-feature extraction of the information to be detected based on the neural network recognition model. The information to be detected is a captured vehicle front view image. The neural network recognition model is trained by a labeled training sample set. The labeled training sample set specifically includes an image set in which objects contained in the image have been labeled. The target detection module is used to detect targets in the vehicle-mounted front view image based on the extracted multi-scale features.

10. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the target detection method according to any one of claims 1-8.

11. A storage medium containing computer-executable instructions, characterized in that, The computer-executable instructions, when executed by a computer processor, are used to perform the target detection method as described in any one of claims 1-8.

Images