Target detection method, apparatus, device, and medium

Abstract

The application discloses a target detection method and device, equipment and medium, and relates to the field of computer vision. The method comprises the following steps: determining a feature combination corresponding to the detection target type of a current to-be-detected image from HOG features, LBP features, GM features, LUV features and SIFT features by using a target detection network, and obtaining multi-scale target traditional features; extracting deep features and target traditional features of the current to-be-detected image by using the target detection network, performing light intensity measurement on the current to-be-detected image by using ISP hardware, obtaining first weights and second weights corresponding to the deep features and the target traditional features respectively, so as to obtain weighted deep features of the deep features and weighted traditional features of the target traditional features, and obtaining labeling information corresponding to the current to-be-detected image by using a feature map obtained by splicing the weighted deep features and the weighted traditional features. Through the above scheme, the quality of target detection can be ensured while the cost is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer vision, in particular to a target detection method, device, equipment and medium. BACKGROUND

[0002] Target detection is an important research hotspot in the field of computer vision, and can be applied to face detection, pedestrian detection, vehicle detection, detection of roads in satellite images, obstacle detection in vehicle camera images, and lesion detection in medical images. For example, in the intelligent driving industry, it mainly uses data information of sensors such as cameras and radars to detect specific targets such as pedestrians, vehicles, and road edges from data through pattern recognition or deep learning algorithms. Target detection is closely related to important functions such as AEB (Autonomous Emergency Braking), ACC (Adaptive Cruise Control), LKS (Lane Keeping System), APA (Auto Parking Assist), and NOA (Navigate on Autopilot) on the vehicle side.

[0003] In the prior art, a convolutional neural network is used to determine the weights of traditional features and deep features respectively, so as to complete target detection using weighted traditional features and weighted deep features. However, using a convolutional neural network to determine feature weights has the following problems: on the one hand, a large amount of data needs to be collected to build a large-scale model, and the required computing resources are large and the cost is high; on the other hand, the weights determined by the convolutional neural network under extreme conditions are uncertain, so the quality of target detection cannot be guaranteed.

[0004] In summary, how to reduce the cost while guaranteeing the quality of target detection is a problem to be solved in the field. SUMMARY

[0005] Therefore, the purpose of the present application is to provide a target detection method, device, equipment and medium, which can reduce the cost while guaranteeing the quality of target detection. The specific solutions are as follows:

[0006] In a first aspect, the present application discloses a target detection method, comprising:

[0007] acquiring a current image to be detected;

[0008] The target detection network based on multi-modal fusion of traditional features and deep features is used to determine a feature combination corresponding to a detection target type of the current image to be detected from HOG features, LBP features, GM features, LUV features and SIFT features, so as to obtain multi-scale target traditional features;

[0009] The target detection network is used to extract deep features of the current image to be detected and the target traditional features, and the ISP hardware is used to measure the light intensity of the current image to be detected, so as to obtain a first weight corresponding to the deep features and a second weight corresponding to the target traditional features;

[0010] The first weight is used to obtain weighted deep features of the deep features, and the second weight is used to obtain weighted traditional features of the target traditional features, and then the weighted deep features and the weighted traditional features are spliced to obtain a feature map of the current image to be detected;

[0011] The feature map is used to obtain annotation information corresponding to the current image to be detected.

[0012] Optionally, before the current image to be detected is collected, the method further comprises:

[0013] An initial target detection network based on deep features is constructed according to a detection target type;

[0014] Multi-scale traditional features are determined based on the detection target type, and the multi-scale traditional features are fused into a backbone network of the initial target detection network, so as to obtain a target detection network based on multi-modal fusion of traditional features and deep features; wherein the multi-scale traditional features are any combination of HOG, LBP, GM, LUV and SIFT features.

[0015] Optionally, the ISP hardware is used to measure the light intensity of the current image to be detected, so as to obtain a first weight corresponding to the deep features and a second weight corresponding to the target traditional features, comprising:

[0016] The global statistical tool in the ISP hardware is used to measure the light intensity of the current image to be detected, so as to obtain the actual light intensity of the current image to be detected;

[0017] A preset expected brightness of the current image to be detected is obtained, the actual light intensity and the preset expected brightness are processed by the automatic exposure tool in the ISP hardware, so as to obtain the exposure intensity of the current image to be detected, and the first weight corresponding to the deep features and the second weight corresponding to the target traditional features are obtained by using the exposure intensity.

[0018] Optionally, concatenating the weighted depth features and the weighted traditional features to obtain the feature map of the current image to be detected includes:

[0019] The weighted depth features and the weighted traditional features are concatenated using the concatenation method to obtain the feature map of the current image to be detected.

[0020] Optionally, the step of obtaining the weighted depth features of the depth features using the first weight, obtaining the weighted traditional features of the target traditional features using the second weight, and then concatenating the weighted depth features and the weighted traditional features includes:

[0021] The weighted depth features of the current layer are obtained using the first weight, and the weighted traditional features of the target traditional features of the current layer are obtained using the second weight.

[0022] The weighted depth features of the current layer and the weighted traditional features of the target traditional features of the current layer are concatenated to obtain the weighted depth features of the depth features of the next layer.

[0023] Optionally, concatenating the weighted depth features of the current layer's depth features and the weighted traditional features of the target traditional features of the current layer to obtain the weighted depth features of the next layer includes:

[0024] The weighted depth features of the current layer and the weighted traditional features of the target traditional features of the current layer are concatenated to obtain the concatenated features of the current layer.

[0025] The target computation logic of the current layer is determined, and the concatenated features of the current layer are processed using the target computation logic of the current layer to obtain the weighted depth features of the depth features of the next layer; wherein, the target computation logic is a set of several convolution, batch normalization and activation functions.

[0026] Optionally, after obtaining the annotation information corresponding to the current image to be detected using the feature map, the process includes:

[0027] Based on the annotation information and the current image to be detected, the corresponding target image is obtained, and the target image is displayed using a preset platform.

[0028] Secondly, this application discloses a target detection device, comprising:

[0029] The image acquisition module is used to acquire the image to be detected.

[0030] The target feature determination module is used to determine the feature combination corresponding to the detection target type of the current image to be detected from HOG features, LBP features, GM features, LUV features and SIFT features using a target detection network based on multimodal fusion of traditional features and deep features, so as to obtain multi-scale traditional target features;

[0031] The weight determination module is used to extract the depth features and the target traditional features of the current image to be detected using the target detection network, and to perform light intensity measurement on the current image to be detected using ISP hardware, so as to obtain the first weight of the depth features and the second weight of the target traditional features.

[0032] The feature map acquisition module is used to acquire the weighted depth features of the depth features using the first weight, and to acquire the weighted traditional features of the target traditional features using the second weight. Then, the weighted depth features and the weighted traditional features are concatenated to obtain the feature map of the current image to be detected.

[0033] The target image acquisition module is used to obtain annotation information corresponding to the current image to be detected using the feature map.

[0034] Thirdly, this application discloses an electronic device, including:

[0035] Memory, used to store computer programs;

[0036] A processor is used to execute the computer program to implement the steps of the aforementioned disclosed target detection method.

[0037] Fourthly, this application discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the steps of the aforementioned disclosed target detection method.

[0038] As can be seen, this application acquires the current image to be detected; utilizes a target detection network based on multimodal fusion of traditional features and depth features to determine feature combinations corresponding to the target type of the current image to be detected from HOG features, LBP features, GM features, LUV features, and SIFT features, so as to obtain multi-scale target traditional features; uses the target detection network to extract the depth features and the target traditional features of the current image to be detected, and uses ISP hardware to perform light intensity measurement on the current image to be detected, so as to obtain the first weight corresponding to the depth features and the second weight corresponding to the target traditional features; uses the first weight to obtain the weighted depth features of the depth features, and uses the second weight to obtain the weighted traditional features of the target traditional features, and then concatenates the weighted depth features and the weighted traditional features to obtain the feature map of the current image to be detected; uses the feature map to obtain the annotation information corresponding to the current image to be detected. Therefore, this application employs a target detection network based on the fusion of traditional and deep features with a smaller scale to detect the current image to be detected, instead of using a traditional convolutional neural network model. This reduces the amount of input data collected, lowers the computational requirements of the hardware, and achieves low-cost target detection. On the other hand, this application uses ISP hardware to measure the light intensity of the current image to be detected to obtain deterministic and quantifiable first and second weights, thereby ensuring the quality of target detection. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0040] Figure 1 This is a flowchart of a target detection method disclosed in this application;

[0041] Figure 2 This application discloses a specific image to be detected.

[0042] Figure 3 This is a schematic diagram of a specific feature fusion disclosed in this application;

[0043] Figure 4 This application discloses a specific target image;

[0044] Figure 5 This is a flowchart of a specific target detection method disclosed in this application;

[0045] Figure 6 This is a schematic diagram of a specific target detection disclosed in this application;

[0046] Figure 7 This is a flowchart of another specific target detection method disclosed in this application;

[0047] Figure 8 This is a schematic diagram of the structure of a target detection device disclosed in this application;

[0048] Figure 9 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0049] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0050] Object detection is a major research hotspot in the field of computer vision, applicable to face detection, pedestrian detection, vehicle detection, road detection in satellite images, obstacle detection in vehicle camera images, and lesion detection in medical images. For example, in the autonomous driving industry, it primarily utilizes data from sensors such as cameras and radar, employing algorithms like pattern recognition or deep learning to detect specific targets such as pedestrians, vehicles, and road edges. Object detection is closely related to crucial vehicle functions such as AEB, ACC, LKS, APA, and NOA.

[0051] In existing technologies, convolutional neural networks are needed to determine the weights of traditional features and deep features separately so that the weighted traditional features and weighted deep features can be used to complete object detection. However, using convolutional neural networks to determine feature weight values ​​requires, on the one hand, a large amount of data to be collected and a large-scale model to be built, and the required computing resources are large and the cost is high; on the other hand, the weights determined by convolutional neural networks under extreme conditions are uncertain, so the quality of object detection cannot be guaranteed.

[0052] Therefore, this application provides a target detection scheme that can reduce costs while ensuring the quality of target detection.

[0053] See Figure 1 As shown in the figure, this application discloses a target detection method, including:

[0054] Step S11: Acquire the current image to be detected.

[0055] For example, in the intelligent driving industry, data such as... are collected while the vehicle is running. Figure 2 The image shown is a specific image to be detected. It is clear that the image to be detected is the original image without any annotation information, which makes it impossible to achieve better intelligent driving.

[0056] In this embodiment, before acquiring the current image to be detected, the method further includes: constructing an initial target detection network based on depth features according to the target type; determining multi-scale traditional features based on the target type, and fusing the multi-scale traditional features into the backbone network of the initial target detection network to obtain a target detection network based on multi-modal fusion of traditional features and depth features; wherein, the multi-scale traditional features are any combination of HOG, LBP, GM, LUV, and SIFT features. The network structure of the fusion method of traditional features and depth features is based on YOLOv5. In the backbone network, the original depth features and traditional image features are aggregated together to construct an ACF (Autoencoder-based Collaborative Filtering)-YOLO Mixed Net network model. It is understood that the traditional features of the multi-scale in this embodiment can be any combination of HOG (Histogram of Oriented Gradient), LBP (Local Binary Patterns), GM (Gradient Magnitude), LUV (Features represented in CIELUV color space), and SIFT (Scale-invariant feature transform). Other traditional features can also be added according to specific circumstances.

[0057] Step S12: Utilize a target detection network based on multimodal fusion of traditional and deep features to determine feature combinations corresponding to the target type of the current image to be detected from HOG features, LBP features, GM features, LUV features, and SIFT features, so as to obtain multi-scale traditional target features.

[0058] When determining multi-scale traditional features, it is necessary to consider the type of target to be detected. Therefore, the fused target detection network is more targeted, and thus, it can achieve excellent target detection results without collecting a large amount of data for training. For example, when detecting pedestrians or vehicles, five artificial features are usually used based on the characteristics of pedestrians and vehicles: HOG, LBP, GM, LUV, and SIFT. Since HOG is calculated in local regions of the image, it has good invariance to geometric and optical deformations of the image. When detecting pedestrians in complex traffic conditions, as long as the pedestrian remains generally upright, movement is allowed. The process involves changes in body movements, and the geometric invariance of HOG features is well-suited to address the challenges of detecting pedestrians in complex image scenarios. Furthermore, it allows for different channel numbers or targeted feature fusion based on the detection target. For example, when detecting speed or weight limits, traditional features are typically added, subtracted, replaced, or strengthened. The red channel of the image might be considered as an added traditional feature, while increasing the proportion of LBP in the traditional features. For instance, GM can effectively assess image blur, LUV can effectively acquire image brightness and chromaticity information, and SIFT has a high tolerance for light and noise.

[0059] In this embodiment, the determined feature combination includes each traditional feature and the corresponding proportion of each traditional feature. For example, the target traditional features of the multi-scale are GM features and SIFT features, and the proportion of GM features is 0.3 and the proportion of SIFT features is 0.7.

[0060] Step S13: Use the target detection network to extract the depth features and the target traditional features of the current image to be detected, and use ISP hardware to measure the light intensity of the current image to be detected to obtain the first weight corresponding to the depth features and the second weight corresponding to the target traditional features.

[0061] It should be noted that in this embodiment, the depth features and target traditional features of the current image to be detected can be extracted first using the object detection network, and then the light intensity of the current image to be detected can be measured using ISP hardware to obtain the first weight corresponding to the depth features and the second weight corresponding to the target traditional features. Alternatively, the light intensity of the current image to be detected can be measured first using ISP hardware to obtain the first weight corresponding to the depth features and the second weight corresponding to the target traditional features, and then the depth features and target traditional features of the current image to be detected can be extracted using the object detection network. Alternatively, the depth features and target traditional features of the current image to be detected can be extracted using the object detection network while the light intensity of the current image to be detected is measured using ISP hardware to obtain the first weight corresponding to the depth features and the second weight corresponding to the target traditional features. In other words, this application does not limit the order of the steps of obtaining depth features and traditional features and the steps of obtaining the first weight and the second weight.

[0062] Because the target traditional features are multi-scale, meaning they contain several traditional features, including HOG features and LBP features, the proportions of the second weights for HOG, LBP, and SIFT features may be the same or different, depending on the specific image to be detected. For example, if the total second weights obtained using ISP (Image Signal Processing) hardware are 0.6 and the first weight is 0.4, and the multi-scale target traditional features are GM features and SIFT features, with GM features accounting for 0.3 and SIFT features for 0.7, then GM features account for 0.18 of all features and SIFT features account for 0.42 of all features.

[0063] Step S14: Use the first weight to obtain the weighted depth features of the depth features, and use the second weight to obtain the weighted traditional features of the target traditional features. Then, concatenate the weighted depth features and the weighted traditional features to obtain the feature map of the current image to be detected.

[0064] For example Figure 3The diagram illustrates a specific feature fusion process. Hyperparameters α and β are introduced as the second weight of the target traditional feature and the first weight of the depth feature, respectively. The product of the target traditional feature and the second weight of the current layer is used as the weighted traditional feature of the current layer, and the product of the depth feature and the first weight of the current layer is used as the weighted depth feature of the current layer. Then, the sum of the weighted traditional feature and the weighted depth feature of the current layer is used as the concatenated feature of the current layer. The concatenated feature of the current layer is then processed using the target computation logic of the current layer to obtain the depth feature of the next layer. The target computation logic (CBLs, i.e., Convolution, Batch Normalization, Leaky ReLU set) is a set of convolution, batch normalization, and activation functions.

[0065] Step S15: Use the feature map to obtain the annotation information corresponding to the current image to be detected.

[0066] In this embodiment, after obtaining the annotation information corresponding to the current image to be detected using the feature map, the method further includes: obtaining the corresponding target image based on the annotation information and the current image to be detected, and displaying the target image using a preset platform. The target detection network is then used to... Figure 2 The current image to be detected is processed to obtain a feature map. The feature map can be used to obtain, for example... Figure 4 The image shown is a specific target image, where the annotation information can be... Figure 4 The target type information and location information of the current image to be detected are obtained by using the target detection network. In other words, the annotation information in this embodiment is not the input information of the target detection network, nor is it information obtained by manual annotation. The target image can be displayed on the vehicle's screen. The target image and annotation information can be applied to functions such as AEB, ACC, LKS, APA, and NOA.

[0067] As can be seen, this application acquires the current image to be detected; utilizes a target detection network based on multimodal fusion of traditional features and depth features to determine feature combinations corresponding to the target type of the current image to be detected from HOG features, LBP features, GM features, LUV features, and SIFT features, so as to obtain multi-scale target traditional features; uses the target detection network to extract the depth features and the target traditional features of the current image to be detected, and uses ISP hardware to perform light intensity measurement on the current image to be detected, so as to obtain the first weight corresponding to the depth features and the second weight corresponding to the target traditional features; uses the first weight to obtain the weighted depth features of the depth features, and uses the second weight to obtain the weighted traditional features of the target traditional features, and then concatenates the weighted depth features and the weighted traditional features to obtain the feature map of the current image to be detected; uses the feature map to obtain the annotation information corresponding to the current image to be detected. Therefore, this application employs a target detection network based on the fusion of traditional and deep features with a smaller scale to detect the current image to be detected, instead of using a traditional convolutional neural network model. This reduces the amount of input data collected, lowers the computational requirements of the hardware, and achieves low-cost target detection. On the other hand, this application uses ISP hardware to measure the light intensity of the current image to be detected to obtain deterministic and quantifiable first and second weights, thereby ensuring the quality of target detection.

[0068] See Figure 5 As shown in the figure, this application discloses a specific target detection method, including:

[0069] Step S21: Acquire the current image to be detected.

[0070] Step S22: Utilize a target detection network based on multimodal fusion of traditional and deep features to determine feature combinations corresponding to the target type of the current image to be detected from HOG features, LBP features, GM features, LUV features, and SIFT features, so as to obtain multi-scale traditional target features.

[0071] Step S23: Use the target detection network to extract the depth features and target traditional features of the current image to be detected, and use ISP hardware to measure the light intensity of the current image to be detected to obtain the first weight corresponding to the depth features and the second weight corresponding to the target traditional features.

[0072] For example Figure 6The diagram illustrates a specific object detection method. It extracts traditional features (HOG, LBP, GM, LUV, SIFT) and depth features from the current image to be detected. The object detection network determines the first weight of the traditional features and the second weight of the depth features based on specific circumstances. Before extracting the depth and traditional features, the current image to be detected can undergo Normalize Gamma & Color correction and Downscale processing. It's understandable that the second weights of the various traditional features determined by the object detection network may be the same or different, depending on the specific image to be detected. For example, when detecting speed or weight limits, the feature combination is HOG, LBP, GM, and LUV. Generally, corresponding traditional features are added, subtracted, replaced, or strengthened. The red channel of the image might be considered as an added traditional feature, while increasing the proportion of LBP in the traditional features. In practical applications, a possible weight allocation is: HOG weight 0.25, LBP weight 0.3, GM weight 0.2, and Red weight 0.25.

[0073] Step S24: Use the first weight to obtain the weighted depth features of the depth features, and use the second weight to obtain the weighted traditional features of the target traditional features. Then use the concatenation method to concatenate the weighted depth features and the weighted traditional features to obtain the feature map of the current image to be detected.

[0074] In this embodiment, the step of obtaining the weighted depth features of the depth features using the first weight, obtaining the weighted traditional features of the target traditional features using the second weight, and then concatenating the weighted depth features and the weighted traditional features includes: obtaining the weighted depth features of the current layer's depth features using the first weight, and obtaining the weighted traditional features of the target traditional features of the current layer using the second weight; concatenating the weighted depth features of the current layer's depth features and the weighted traditional features of the target traditional features of the current layer to obtain the weighted depth features of the next layer's depth features. That is, hyperparameters α and β are introduced as the second weight of the target traditional features and the first weight of the depth features, respectively. The product of the target traditional features and the second weight of the current layer is used as the weighted traditional features of the current layer, and the product of the current layer's depth features and the first weight is used as the weighted depth features of the current layer. The sum of the weighted traditional features and the weighted depth features of the current layer is used as the concatenated features of the current layer. The concatenated features of the current layer are then processed using the target calculation logic of the current layer to obtain the depth features of the next layer.

[0075] In this embodiment, concatenating the weighted depth features of the current layer's depth features and the weighted traditional features of the target traditional features of the current layer to obtain the weighted depth features of the next layer includes: concatenating the weighted depth features of the current layer's depth features and the weighted traditional features of the target traditional features of the current layer to obtain the concatenated features of the current layer; determining the target computation logic of the current layer, and processing the concatenated features of the current layer using the target computation logic of the current layer to obtain the weighted depth features of the next layer's depth features; wherein, the target computation logic is a set of several convolutions, batch normalizations, and activation functions. It can be understood that after obtaining the depth features of the last layer, processing the depth features of the last layer using the target computation logic of the last layer, and then performing corresponding convolution processing, a feature map can be obtained.

[0076] Step S25: Use the feature map to obtain the annotation information corresponding to the current image to be detected.

[0077] It is understandable that the obtained feature map needs to be decoded, so... Figure 6 As shown, the post-processing decoding unit processes the feature map to obtain the processed feature map. Only by using the processed feature map can the corresponding annotation information of the current image to be detected be obtained.

[0078] Therefore, it can be seen that the fusion of traditional features of the target in this application can enable the target detection network to output better target detection results without the need for network training with a large amount of data, thereby reducing the cost of collecting, filtering and calibrating input data; the small model reduces the hardware requirements for storage and computing power, thereby improving the product's competitiveness in the market; and the reduction in computing power consumption brings great benefits to the stability of the product.

[0079] See Figure 7 As shown in the figure, this application discloses a specific target detection method, including:

[0080] Step S31: Acquire the current image to be detected.

[0081] Step S32: Utilize a target detection network based on multimodal fusion of traditional and deep features to determine feature combinations corresponding to the target type of the current image to be detected from HOG features, LBP features, GM features, LUV features, and SIFT features, so as to obtain multi-scale traditional target features.

[0082] Step S33: Use the target detection network to extract the depth features of the current image to be detected and the target traditional features.

[0083] Step S34: Use the global statistical tool in the ISP hardware to measure the light intensity of the current image to be detected, and obtain the actual light intensity of the current image to be detected; obtain the preset expected brightness of the current image to be detected, and use the automatic exposure tool in the ISP hardware to process the actual light intensity and the preset expected brightness to obtain the exposure intensity of the current image to be detected, and use the exposure intensity to obtain the first weight corresponding to the depth feature and the second weight corresponding to the target traditional feature.

[0084] In this embodiment, the user can preset the desired brightness, that is, obtain the preset desired brightness Lt (Luma target) of the current image to be detected. After inputting the current image to be detected Is (Image Source) into the Automatic Exposure (AE) tool, the global statistical tool in the ISP hardware measures the light intensity of the current image to be detected Is to obtain the actual light intensity of the current image to be detected. In this way, the automatic exposure tool can process the actual light intensity and the preset desired brightness to obtain the exposure intensity of the current image to be detected. The specific calculation formula is as follows:

[0085] Ev = FAE(Is, Lt);

[0086] In the formula, Ev represents the exposure intensity, and the range of Ev is (Min(Ev), Max(Ev)). FAE represents the algorithm of the automatic exposure tool, Is represents the current image to be detected, and Lt represents the preset expected brightness, which is usually 130 to 140.

[0087] After obtaining the exposure intensity Ev, it is also necessary to obtain the normalized value Evn of the exposure intensity Ev. Using Evn, the first weight β corresponding to the depth feature and the second weight α corresponding to the target traditional feature are obtained. The specific calculation formula is as follows:

[0088] α=(1-2C)×Evn 2 +C;

[0089] β = 1 - α;

[0090] In the formula, α represents the second weight corresponding to the traditional feature of the target, β represents the first weight corresponding to the depth feature, Evn is the normalized value of the exposure intensity Ev, which is in the range of (-1, 1), and C represents the adjustment parameter, which is in the range of 0-0.5.

[0091] Step S35: Obtain the weighted depth features of the depth features using the first weight, and obtain the weighted traditional features of the target traditional features using the second weight. Then, concatenate the weighted depth features and the weighted traditional features to obtain the feature map of the current image to be detected.

[0092] Step S36: Use the feature map to obtain the annotation information corresponding to the current image to be detected.

[0093] As can be seen, this application combines traditional features and deep features. Traditional features rely more on the quality of the image to be detected. Therefore, the relevant parameters can be adaptively adjusted by the ISP hardware to obtain deterministic and quantifiable weights, which makes up for the shortcomings of traditional features under extreme conditions, gives full play to the advantages of existing hardware, and ensures the effect of target detection.

[0094] See Figure 8 As shown in the figure, this application discloses a target detection device, including:

[0095] Image acquisition module 11 is used to acquire the current image to be detected;

[0096] The target feature determination module 12 is used to determine the feature combination corresponding to the detection target type of the current image to be detected from HOG features, LBP features, GM features, LUV features and SIFT features using a target detection network based on multimodal fusion of traditional features and deep features, so as to obtain multi-scale traditional target features.

[0097] The weight determination module 13 is used to extract the depth features and the target traditional features of the current image to be detected using the target detection network, and to perform light intensity measurement on the current image to be detected using ISP hardware, so as to obtain the first weight of the depth features and the second weight of the target traditional features.

[0098] The feature map acquisition module 14 is used to acquire the weighted depth features of the depth features using the first weight, and to acquire the weighted traditional features of the target traditional features using the second weight, and then to concatenate the weighted depth features and the weighted traditional features to obtain the feature map of the current image to be detected.

[0099] The target image acquisition module 15 is used to obtain annotation information corresponding to the current image to be detected using the feature map.

[0100] As can be seen, this application acquires the current image to be detected; utilizes a target detection network based on multimodal fusion of traditional features and depth features to determine feature combinations corresponding to the target type of the current image to be detected from HOG features, LBP features, GM features, LUV features, and SIFT features, so as to obtain multi-scale target traditional features; uses the target detection network to extract the depth features and the target traditional features of the current image to be detected, and uses ISP hardware to perform light intensity measurement on the current image to be detected, so as to obtain the first weight corresponding to the depth features and the second weight corresponding to the target traditional features; uses the first weight to obtain the weighted depth features of the depth features, and uses the second weight to obtain the weighted traditional features of the target traditional features, and then concatenates the weighted depth features and the weighted traditional features to obtain the feature map of the current image to be detected; uses the feature map to obtain the annotation information corresponding to the current image to be detected. Therefore, this application employs a target detection network based on the fusion of traditional and deep features with a smaller scale to detect the current image to be detected, instead of using a traditional convolutional neural network model. This reduces the amount of input data collected, lowers the computational requirements of the hardware, and achieves low-cost target detection. On the other hand, this application uses ISP hardware to measure the light intensity of the current image to be detected to obtain deterministic and quantifiable first and second weights, thereby ensuring the quality of target detection.

[0101] Furthermore, embodiments of this application also provide an electronic device. Figure 9 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application.

[0102] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Specifically, it may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the target detection method performed by the electronic device disclosed in any of the foregoing embodiments.

[0103] In this embodiment, the power supply 23 is used to provide operating voltage for various hardware devices on the electronic device; the communication interface 24 can create a data transmission channel between the electronic device and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0104] The processor 21 may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor 21 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 21 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 21 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, the processor 21 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0105] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored on it include operating system 221, computer program 222 and data 223, etc., and the storage method can be temporary storage or permanent storage.

[0106] The operating system 221 manages and controls the various hardware devices and computer programs 222 on the electronic device to enable the processor 21 to perform calculations and processing on the massive amounts of data 223 in the memory 22. The operating system can be Windows, Unix, Linux, etc. The computer program 222, in addition to including a computer program capable of performing the target detection method executed by the electronic device as disclosed in any of the foregoing embodiments, may further include computer programs capable of performing other specific tasks. The data 223 may include data received by the electronic device from external devices, as well as data collected by its own input / output interface 25.

[0107] Furthermore, embodiments of this application also disclose a computer-readable storage medium storing a computer program. When the computer program is loaded and executed by a processor, it implements the method steps performed during the target detection process disclosed in any of the foregoing embodiments.

[0108] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0109] The above provides a detailed description of the target detection method, apparatus, device, and medium provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A target detection method, characterized in that, include: Acquire the current image to be detected; A multimodal target detection network based on the fusion of traditional and deep features is used to determine the feature combination corresponding to the target type of the current image to be detected from HOG features, LBP features, GM features, LUV features, and SIFT features, so as to obtain multi-scale traditional target features; wherein, the target detection network is the ACF-Yolo Mixed Net network model; The target detection network is used to extract the depth features and the target traditional features of the current image to be detected, and the light intensity of the current image to be detected is measured using ISP hardware to obtain the first weight corresponding to the depth features and the second weight corresponding to the target traditional features. The weighted depth features of the depth features are obtained using the first weight, and the weighted traditional features of the target traditional features are obtained using the second weight. Then, the weighted depth features and the weighted traditional features are concatenated to obtain the feature map of the current image to be detected. The feature map is used to obtain the annotation information corresponding to the current image to be detected; The step of using ISP hardware to measure the light intensity of the current image to be detected, in order to obtain the first weight corresponding to the depth feature and the second weight corresponding to the target traditional feature, includes: The light intensity of the current image to be detected is measured using the global statistical tool in the ISP hardware to obtain the actual light intensity of the current image to be detected; the preset expected brightness of the current image to be detected is obtained; the actual light intensity and the preset expected brightness are processed using the automatic exposure tool in the ISP hardware to obtain the exposure intensity of the current image to be detected; and the normalized exposure intensity is used to obtain the first weight corresponding to the depth feature and the second weight corresponding to the target traditional feature.

2. The target detection method according to claim 1, characterized in that, Before acquiring the current image to be detected, the process also includes: Construct an initial target detection network based on deep features according to the type of target to be detected; Based on the target type, multi-scale traditional features are determined, and the multi-scale traditional features are fused into the backbone network of the initial target detection network to obtain a target detection network based on multi-modal fusion of traditional features and deep features; wherein, the multi-scale traditional features are any combination of several features among HOG, LBP, GM, LUV, and SIFT features.

3. The target detection method according to claim 1, characterized in that, The step of concatenating the weighted depth features and the weighted traditional features to obtain the feature map of the current image to be detected includes: The weighted depth features and the weighted traditional features are concatenated using the concatenation method to obtain the feature map of the current image to be detected.

4. The target detection method according to claim 1, characterized in that, The step of obtaining a weighted depth feature of the depth feature using the first weight, obtaining a weighted traditional feature of the target traditional feature using the second weight, and then concatenating the weighted depth feature and the weighted traditional feature includes: The weighted depth features of the current layer are obtained using the first weight, and the weighted traditional features of the target traditional features of the current layer are obtained using the second weight. The weighted depth features of the current layer and the weighted traditional features of the target traditional features of the current layer are concatenated to obtain the weighted depth features of the depth features of the next layer.

5. The target detection method according to claim 4, characterized in that, The step of concatenating the weighted depth features of the current layer's depth features and the weighted traditional features of the target traditional features of the current layer to obtain the weighted depth features of the next layer includes: The weighted depth features of the current layer and the weighted traditional features of the target traditional features of the current layer are concatenated to obtain the concatenated features of the current layer. The target computation logic of the current layer is determined, and the concatenated features of the current layer are processed using the target computation logic of the current layer to obtain the weighted depth features of the depth features of the next layer; wherein, the target computation logic is a set of several convolution, batch normalization and activation functions.

6. The target detection method according to any one of claims 1 to 5, characterized in that, After obtaining the annotation information corresponding to the current image to be detected using the feature map, the process includes: Based on the annotation information and the current image to be detected, the corresponding target image is obtained, and the target image is displayed using a preset platform.

7. A target detection device, characterized in that, include: The image acquisition module is used to acquire the image to be detected. The target feature determination module is used to determine the feature combination corresponding to the target type of the current image to be detected from HOG features, LBP features, GM features, LUV features, and SIFT features using a target detection network based on multimodal fusion of traditional features and deep features, so as to obtain multi-scale traditional target features; wherein, the target detection network is an ACF-YoloMixed Net network model; The weight determination module is used to extract the depth features and the target traditional features of the current image to be detected using the target detection network, and to perform light intensity measurement on the current image to be detected using ISP hardware, so as to obtain the first weight of the depth features and the second weight of the target traditional features. The feature map acquisition module is used to acquire the weighted depth features of the depth features using the first weight, and to acquire the weighted traditional features of the target traditional features using the second weight. Then, the weighted depth features and the weighted traditional features are concatenated to obtain the feature map of the current image to be detected. The target image acquisition module is used to obtain annotation information corresponding to the current image to be detected using the feature map; The weight determination module is specifically used for: The light intensity of the current image to be detected is measured using the global statistical tool in the ISP hardware to obtain the actual light intensity of the current image to be detected; the preset expected brightness of the current image to be detected is obtained; the actual light intensity and the preset expected brightness are processed using the automatic exposure tool in the ISP hardware to obtain the exposure intensity of the current image to be detected; and the normalized exposure intensity is used to obtain the first weight corresponding to the depth feature and the second weight corresponding to the target traditional feature.

8. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the target detection method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, Used to store a computer program; wherein, when the computer program is executed by a processor, it implements the steps of the target detection method as described in any one of claims 1 to 6.

Images