Scene object detection method, system and electronic device based on deep learning

Abstract

The present application relates to the field of target detection, and specifically discloses a scene target detection method, system and electronic device based on deep learning, which adopts artificial intelligence detection technology based on machine vision, obtains the vehicle surrounding environment image during the driving process through the vehicle-mounted camera, and extracts the features of the vehicle surrounding environment image in high-dimensional space to analyze and determine whether there is a target object. In this way, real-time target detection can be achieved during the driving process, which helps to perceive and understand the road environment more accurately.

Description

Technical Field

[0001] The present application relates to the field of target detection, and more specifically, to a scene target detection method, system and electronic device based on deep learning. Background Art

[0002] With the continuous development of the economy and society, cars, as an important means of transportation, have brought great convenience to people's travel. However, with the rapid increase in the number of cars, a series of related problems have also arisen. Frequent accidents have become a worrying issue. A large number of traffic accidents occur every year, posing a huge threat to people's lives and property. At the same time, traffic congestion is becoming increasingly serious, especially in the central areas of cities, where vehicles often queue up.

[0003] In order to solve these traffic problems, autonomous driving technology has become the focus of attention. Autonomous driving cars can autonomously perceive the surrounding road environment while driving and make corresponding decisions and actions based on the perceived information. The development of this technology has received extensive attention and investment from many countries and companies.

[0004] In the environmental perception system of self-driving cars, object detection is a very critical component. It analyzes and processes the received road information to provide accurate data for the decision-making system to ensure driving safety. However, self-driving cars face complex and changing road environments, where there are many small and possibly obscured targets that are difficult to detect. Traditional object detection algorithms perform poorly in these situations and are difficult to achieve accuracy and stability requirements.

[0005] Therefore, an optimized deep learning-based scene object detection scheme is needed. Summary of the invention

[0006] In order to solve the above technical problems, this application is proposed. The embodiments of this application provide a scene target detection method, system and electronic device based on deep learning, which adopts artificial intelligence detection technology based on machine vision, obtains the vehicle surrounding environment image during the driving process of the car through the on-board camera, and extracts the features of the vehicle surrounding environment image in high-dimensional space to analyze and determine whether there is a target object. In this way, real-time target detection can be achieved during the driving process of the car, which helps to perceive and understand the road environment more accurately.

[0007] According to one aspect of the present application, a scene object detection method based on deep learning is provided, which includes:

[0008] Acquire an image of the vehicle's surroundings captured by a vehicle-mounted camera;

[0009] Performing image preprocessing on the vehicle surrounding environment image to obtain a preprocessed vehicle surrounding environment image;

[0010] Passing the preprocessed vehicle surrounding environment image through an environment feature extractor based on a hybrid convolutional layer to obtain a plurality of vehicle environment feature maps;

[0011] fusing the plurality of vehicle environment characteristic maps to obtain a vehicle environment comprehensive characteristic map;

[0012] Input the vehicle environment comprehensive feature map into the Shuffle NetV2 basic module to obtain a target detection classification feature map;

[0013] Optimizing the target detection classification feature map to obtain an optimized target detection classification feature map;

[0014] The optimized target detection classification feature map is passed through a classifier to obtain a classification result, and the classification result is used to indicate whether a target object exists.

[0015] In the above-mentioned scene target detection method based on deep learning, the hybrid convolution layer includes a parallel first convolution branch structure, a second convolution branch structure, a third convolution branch structure and a fourth convolution branch structure, wherein the first convolution branch uses a first convolution kernel having a first size, the second convolution branch uses a second convolution kernel having a first size and a first dilation rate, the third convolution branch uses a third convolution kernel having a first size and a second dilation rate, and the fourth convolution branch uses a fourth convolution kernel having a first size and a third dilation rate.

[0016] In the above-mentioned scene target detection method based on deep learning, the preprocessed vehicle surrounding environment image is passed through an environment feature extractor based on a hybrid convolutional layer to obtain multiple vehicle environment feature maps, including: using the first convolution kernel with a first size to perform convolution encoding on the preprocessed vehicle surrounding environment image to obtain a first-scale vehicle environment feature map; using the second convolution kernel with a first hole rate to perform convolution encoding on the preprocessed vehicle surrounding environment image to obtain a second-scale vehicle environment feature map; using the third convolution kernel with a second hole rate to perform convolution encoding on the preprocessed vehicle surrounding environment image to obtain a third-scale vehicle environment feature map; using the fourth convolution kernel with a third hole rate to perform convolution encoding on the preprocessed vehicle surrounding environment image to obtain a fourth-scale vehicle environment feature map.

[0017] In the above-mentioned scene target detection method based on deep learning, the optimization of the target detection classification feature map to obtain the optimized target detection classification feature map includes: calculating the latent feature expression of the motion parameterized model of the target detection classification feature map relative to the target classification function by the following optimization formula to obtain the optimized target detection classification feature map; wherein the optimization formula is:

[0018]

[0019] Among them, f i，j，k represents the feature value of the (i, j, k)th position of the target detection classification feature map, log represents the logarithmic function value with base 2, λ represents a predetermined hyperparameter, and f i，j，k ′ represents the feature value of the (i, j, k)th position of the optimized target detection classification feature map.

[0020] In the above-mentioned scene target detection method based on deep learning, the step of passing the optimized target detection classification feature map through a classifier to obtain a classification result, and the classification result is used to indicate whether a target object exists, comprising: using the classifier to process the optimized target detection classification feature map using the following classification formula to generate the classification result; wherein the classification formula is:

[0021] O=softmax{(W n , B n ):…:(W1,B1)|Project(F)}

[0022] Where O is the classification result, Project(F) represents the projection of the optimized target detection classification feature map into a vector, W1 to W n is the weight matrix of each fully connected layer, B1 to B n Represents the bias vector of each fully connected layer, and softmax is a normalized exponential function.

[0023] According to another aspect of the present application, a scene object detection system based on deep learning is provided, comprising:

[0024] A vehicle surrounding environment data acquisition module is used to acquire the vehicle surrounding environment image taken by the vehicle-mounted camera;

[0025] An environment image preprocessing module, used for performing image preprocessing on the vehicle surrounding environment image to obtain a preprocessed vehicle surrounding environment image;

[0026] An environmental feature extraction module, used for passing the preprocessed vehicle surrounding environment image through an environmental feature extractor based on a hybrid convolutional layer to obtain a plurality of vehicle environment feature maps;

[0027] An environmental feature fusion module, used for fusing the multiple vehicle environmental feature maps to obtain a comprehensive vehicle environmental feature map;

[0028] A target detection feature generation module, used for inputting the vehicle environment comprehensive feature map into the Shuffle NetV2 basic module to obtain a target detection classification feature map;

[0029] An optimization module, used for optimizing the target detection classification feature map to obtain an optimized target detection classification feature map;

[0030] The target detection result generation module is used to pass the optimized target detection classification feature map through a classifier to obtain a classification result, and the classification result is used to indicate whether a target object exists.

[0031] According to another aspect of the present application, an electronic device is provided, comprising a memory and a processor coupled to the memory, wherein the processor is configured to execute the deep learning-based scene target detection method as described above based on instructions stored in the memory.

[0032] Compared with the prior art, the scene target detection method, system and electronic device based on deep learning provided by the present application adopt artificial intelligence detection technology based on machine vision, and obtain the vehicle surrounding environment image during the driving process through the vehicle-mounted camera, and perform high-dimensional space feature extraction on the vehicle surrounding environment image to analyze and determine whether there is a target object. In this way, real-time target detection can be achieved during the driving process, which helps to perceive and understand the road environment more accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0034] Figure 1 Flowchart of a scene target detection method based on deep learning according to an embodiment of the present application.

[0035] Figure 2 It is an architecture diagram of a scene object detection method based on deep learning according to an embodiment of the present application.

[0036] Figure 3 The present invention is a flowchart of obtaining a plurality of vehicle environment feature maps by passing the preprocessed vehicle surrounding environment image through an environment feature extractor based on a hybrid convolutional layer in a scene target detection method based on deep learning according to an embodiment of the present application.

[0037] Figure 4 4 is a system block diagram of a scene object detection system based on deep learning according to an embodiment of the present application.

[0038] Figure 5 is a block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. These embodiments are provided to make the present application comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art. Here, the illustrative embodiments of the present application and their description are used to explain the present application, but are not intended to limit the present application.

[0040] As mentioned in the above background technology, with the development of social economy, cars, as an important means of transportation, have greatly facilitated people's travel. However, with the rapid increase in the number of cars, a series of related problems have also followed. Among them, the frequent occurrence of traffic accidents is worrying, and a large number of accidents every year pose a huge threat to people's lives and property safety. At the same time, traffic congestion is becoming increasingly serious, especially in the central area of ​​the city, where vehicles often queue up. In order to solve these traffic problems, autonomous driving technology has become a hot topic. Autonomous driving is an integrated system that integrates many high-tech technologies. Its key technologies include environmental perception, logical reasoning and decision-making, motion control, processor performance, etc. With the continuous advancement and development of technology, autonomous driving will play an increasingly important role in the future transportation field. Environmental perception is a key part of the autonomous driving system, which includes the vehicle's perception and understanding of the surrounding environment. Autonomous driving vehicles can independently perceive the surrounding road environment during driving and make corresponding decisions and actions based on the perceived information. The development of this technology has attracted widespread attention and investment from many countries and companies. In the environmental perception system of autonomous driving vehicles, target detection is a crucial component. It provides accurate data to the decision-making system by analyzing and processing the received road information to ensure driving safety, such as identifying the location and movement of traffic signals, obstacles, pedestrians, and other vehicles. However, autonomous vehicles face complex and changing road environments, in which there are many small and possibly obscured targets that are difficult to detect. Traditional target detection algorithms perform poorly in these situations and are difficult to meet the requirements of accuracy and stability. Therefore, an optimized scene target detection solution based on deep learning is expected.

[0041] In recent years, deep learning and neural networks have been widely used in computer vision, natural language processing, text signal processing and other fields. In addition, deep learning and neural networks have also demonstrated a level close to or even beyond that of humans in image classification, object detection, semantic segmentation, text translation and other fields. The development of deep learning and neural networks has provided new solutions and solutions for scene object detection based on deep learning.

[0042] The vehicle surrounding environment image contains the shape information, color information, texture information, motion information, etc. of the objects in the vehicle surrounding environment. The technical concept of the present application is to pre-process the acquired vehicle surrounding environment image first, then use a feature extractor to extract relevant vehicle surrounding environment features from the image, and then use the extracted features through a classifier to determine whether there is a target object.

[0043] Figure 1 Flowchart of a scene target detection method based on deep learning according to an embodiment of the present application. Figure 2 FIG. 1 is an architecture diagram of a scene object detection method based on deep learning according to an embodiment of the present application. Figure 1 and Figure 2 As shown, according to the deep learning-based scene target detection method of the embodiment of the present application, the method includes: S110, acquiring a vehicle surrounding environment image taken by a vehicle-mounted camera; S120, performing image preprocessing on the vehicle surrounding environment image to obtain a preprocessed vehicle surrounding environment image; S130, passing the preprocessed vehicle surrounding environment image through an environment feature extractor based on a hybrid convolutional layer to obtain a plurality of vehicle environment feature maps; S140, fusing the plurality of vehicle environment feature maps to obtain a vehicle environment comprehensive feature map; S150, inputting the vehicle environment comprehensive feature map into the Shuffle NetV2 basic module to obtain a target detection classification feature map; S160, optimizing the target detection classification feature map to obtain an optimized target detection classification feature map; S170, passing the optimized target detection classification feature map through a classifier to obtain a classification result, and the classification result is used to indicate whether a target object exists.

[0044] Specifically, in the technical solution of the present application, first, an image of the vehicle's surroundings taken by a vehicle-mounted camera is obtained. It is not difficult to understand that the vehicle-mounted camera can provide a true perception of the vehicle's surroundings. By acquiring environmental images, real-time, high-resolution visual information can be obtained, which can reflect the actual road conditions and the position, shape, and motion state of surrounding objects. This is crucial for an autonomous driving system because it needs to accurately perceive the surrounding environment to make correct decisions. In autonomous driving scenarios, real-time and low latency are very important because the vehicle needs to perceive the surrounding environment and respond in a timely manner. The vehicle-mounted camera can provide real-time image data. By directly acquiring images from the vehicle-mounted camera, the processing and transmission delays of sensor data can be reduced, thereby improving the real-time performance of the system.

[0045] There may be various noises and interferences in the images captured by the vehicle camera, such as image blur, illumination changes, lens distortion, etc. These factors may affect the performance of the target detection algorithm. Therefore, the vehicle surrounding environment image is subjected to image preprocessing to obtain a preprocessed vehicle surrounding environment image. By performing image preprocessing on the vehicle surrounding environment image, the noise and interference in the vehicle surrounding environment image can be reduced, the image quality can be improved, and it is helpful to detect the target more accurately, and it is also helpful to improve the visibility and recognition of the target, making the target easier to detect.

[0046] In an embodiment of the present application, a possible implementation method for performing image preprocessing on the vehicle surrounding environment image to obtain a preprocessed vehicle surrounding environment image may be: 1. reading the original vehicle surrounding environment image; 2. applying a Gaussian filter to perform image smoothing to remove high-frequency noise and smooth the image, and considering applying a deblurring algorithm to reduce image blur; 3. selecting a suitable histogram equalization to enhance the image contrast according to demand, which helps to highlight the boundaries and details of the target, making the target more obvious and easier to detect. You can try to use the CLAHE (Contrast Limited Adaptive Histogram Equalization) algorithm, which can maintain the local contrast of the image; 4. performing scale normalization processing on the image subjected to the above steps and scaling it to a uniform scale. Bilinear interpolation or nearest neighbor interpolation and other methods may be used to perform image scaling operations.

[0047] The preprocessed vehicle surrounding environment image contains rich information, and it is necessary to extract features related to the target object from the image. That is, the preprocessed vehicle surrounding environment image is passed through the environment feature extractor based on the hybrid convolution layer to obtain multiple vehicle environment feature maps. By inputting the preprocessed vehicle surrounding environment image into the environment feature extractor of the hybrid convolution layer, the hybrid convolution layer can capture the structure, texture, color and other information of the vehicle surrounding environment. The convolution layer is a commonly used layer type in deep learning. It performs filtering operations on the input image in a sliding manner to extract local features in the image. The convolution layer consists of a set of learnable convolution kernels (also called filters), each of which can extract different features. Specifically, the convolution layer multiplies the convolution kernel by element-by-element with different positions of the input image, and adds the multiplication results to obtain the output feature map. This process can be regarded as the process of sliding the convolution kernel on the input image and extracting local features. The parameters of the convolution kernel are learnable, and the best feature extraction method is automatically learned through the training process. The hybrid convolution layer contains multiple convolution kernels of different scales, which can extract features at different scales. This can capture environmental features of different scales, so that the target detection algorithm can effectively detect targets of different scales. For example, a smaller convolution kernel can capture detailed features, while a larger convolution kernel can capture overall structural features.

[0048] Specifically, the hybrid convolution layer includes a parallel first convolution branch structure, a second convolution branch structure, a third convolution branch structure, and a fourth convolution branch structure, wherein the first convolution branch uses a first convolution kernel having a first size, the second convolution branch uses a second convolution kernel having a first size and a first void ratio, the third convolution branch uses a third convolution kernel having a first size and a second void ratio, and the fourth convolution branch uses a fourth convolution kernel having a first size and a third void ratio. In the embodiment of the present application, the convolution kernel size of the first size is 3×3.

[0049] Figure 3 The present invention is a flowchart of obtaining a plurality of vehicle environment feature maps by passing the pre-processed vehicle surrounding environment image through an environment feature extractor based on a hybrid convolution layer in a scene target detection method based on deep learning according to an embodiment of the present application. Figure 3As shown, the preprocessed vehicle surrounding environment image is passed through an environment feature extractor based on a hybrid convolutional layer to obtain a plurality of vehicle environment feature maps, including: S131, using the first convolution kernel with a first size to perform convolution encoding on the preprocessed vehicle surrounding environment image to obtain a first-scale vehicle environment feature map; S132, using the second convolution kernel with a first hole rate to perform convolution encoding on the preprocessed vehicle surrounding environment image to obtain a second-scale vehicle environment feature map; S133, using the third convolution kernel with a second hole rate to perform convolution encoding on the preprocessed vehicle surrounding environment image to obtain a third-scale vehicle environment feature map; S134, using the fourth convolution kernel with a third hole rate to perform convolution encoding on the preprocessed vehicle surrounding environment image to obtain a fourth-scale vehicle environment feature map.

[0050] It should be understood that each vehicle environment feature map contains feature information at different levels and scales. By fusing them together, the feature representations at different levels and scales can be comprehensively considered, thereby better capturing the diversity and complexity of the target object. In the technical solution of the present application, the multiple vehicle environment feature maps are fused to obtain a comprehensive vehicle environment feature map. There are many ways to fuse multiple vehicle environment feature maps, such as using convolution operations, weighted summation, etc. The purpose of this is to complement and enhance the useful information in different feature maps to improve the accuracy and robustness of target detection.

[0051] In an embodiment of the present application, a feasible implementation method for fusing the multiple vehicle environment feature maps to obtain a comprehensive vehicle environment feature map is as follows: there are 4 vehicle environment feature maps, namely F1, F2, F3, and F4. The size of each feature map is H×W×C, where H represents the height, W represents the width, and C represents the number of channels. Multiple feature maps can be fused using a weighted summation method. The specific steps are as follows: assign a weight to each feature map to indicate its importance in the fusion. The weight can be set according to the quality, importance, or other prior knowledge of the feature map. Assuming the weights are w1, w2, w3, and w4, the weights are normalized so that they satisfy the condition that the sum is 1. That is, w1'=w1 / (w1+w2+w3+w4), w2'=w2 / (w1+w2+w3+w4), ..., w4'=w4 / (w1+w2+w3+w4); for each feature map Fi, multiply it with the corresponding normalized weight wi', and add them to obtain the fused feature map Ffusion, Ffusion=w1'*F1+w2'*F2+...+w4'*F4; the fused feature map Ffusion is the comprehensive feature map of the vehicle environment, and its size is the same as the input feature map, which is H×W×C.

[0052] After obtaining the comprehensive feature map of the vehicle environment, the features can be further processed and extracted through the deep learning network to provide a more accurate feature representation for the target detection task. That is, the comprehensive feature map of the vehicle environment is input into the Shuffle NetV2 basic module to obtain the target detection classification feature map. ShuffleNetV2 is a lightweight convolutional neural network structure with low computational complexity and parameter quantity, which is suitable for target detection in a resource-constrained environment. By inputting the comprehensive feature map of the vehicle environment into the ShuffleNetV2 basic module, its convolution operation and feature map reorganization capabilities can be used to further extract and integrate feature information. The ShuffleNetV2 basic module can effectively reduce the dimension of the feature map and reduce the computational complexity while maintaining the feature information. It can extract more discriminative feature representations, which helps to distinguish target objects of different categories. The lightweight characteristics of ShuffleNetV2 can also ensure the requirements of real-time target detection under limited computing resources. Such a design can effectively combine deep learning and lightweight network models to achieve efficient and accurate target detection functions.

[0053] The ShuffleNetV2 basic module adopts technologies such as channel-by-channel grouped convolution and channel rearrangement, which can significantly reduce the number of parameters and calculations of the model while maintaining high accuracy. In an embodiment of the present application, a feasible implementation method of inputting the vehicle environment comprehensive feature map into the Shuffle NetV2 basic module to obtain a target detection classification feature map can be: first, the input vehicle environment comprehensive feature map is divided into several groups according to the channel, assuming that the number of channels C is divided into G groups, each group contains C / G channels, and for each group, an independent convolution operation is performed on it. Standard convolution operations or depth-separable convolutions can be used to process each group, so that G output feature maps are obtained, each with C / G channels. Next, the output feature map is subjected to a channel rearrangement operation to interlace the channels of each group together. For each output feature map, its channels can be rearranged according to certain rules, and the specific arrangement rules can be determined according to actual needs. The channel rearrangement operation can be implemented using simple tensor operations, such as using operations such as reshape and transpose. Through channel-by-channel grouped convolution and channel rearrangement operations, the vehicle environment comprehensive feature map can be processed into a target detection classification feature map with a smaller number of channels.

[0054] In particular, in the technical solution of the present application, considering that multiple vehicle environment feature maps are extracted by an environment feature extractor based on a hybrid convolutional layer, there are differences in the feature distribution of multiple vehicle environment feature maps in a high-dimensional feature space. Specifically, the environment feature extractor based on the hybrid convolutional layer extracts features of the vehicle surrounding environment image at different levels, and these features contain information at different abstract levels. The earlier layers may extract some low-level detail features, such as edges, textures, etc., while the deeper layers will extract higher-level semantic features, such as the shape and structure of the object. Due to the different positions and structures of feature extractors at different levels in the network, their perception range and perception ability of the vehicle surrounding environment are also different. The earlier layers are closer to the input image and are more sensitive to detail information and local features; while the deeper layers pay more attention to global semantic information. When multiple vehicle environment feature maps are fused, due to their distribution differences in the feature space, the fused feature map may have structural collapse in the local feature distribution. This means that some detail information may be lost or blurred, thereby affecting the accuracy of the target detection classification feature map obtained by the Shuffle NetV2 basic module. In addition, the Shuffle NetV2 basic module may further change the distribution and structure of the feature map when performing feature processing. This may lead to further structural collapse of the feature map in terms of local feature distribution, thereby affecting the accuracy of the classification results obtained by the classifier. In order to solve the problem of feature distribution differences of multiple vehicle environment feature maps in high-dimensional feature space and structural changes introduced by fusion and Shuffle NetV2 basic modules, in the technical solution of the present application, the implicit feature expression of the motion parameterized model of the target classification function relative to the target classification function is calculated to obtain an optimized target detection classification feature map.

[0055] Specifically, optimizing the target detection classification feature map to obtain an optimized target detection classification feature map includes: calculating the latent feature expression of the motion parameterized model of the target detection classification feature map relative to the target classification function using the following optimization formula to obtain the optimized target detection classification feature map; wherein the optimization formula is:

[0056]

[0057] Among them, f i，j，k represents the feature value of the (i, j, k)th position of the target detection classification feature map, log represents the logarithmic function value with base 2, λ represents a predetermined hyperparameter, and f i，j，k ′ represents the feature value of the (i, j, k)th position of the optimized target detection classification feature map.

[0058] That is, in response to the above technical problems, an optimization method based on latent feature expression is proposed, which can calculate the latent feature expression of the motion parameterization model of the target detection classification feature map relative to the target classification function to obtain the optimized target detection classification feature map, so that by interpreting the eigenvalues ​​of the target detection classification feature map by position probabilistically, the feature distribution extracted by the neural network can be self-adapted with iteration, so as to gradually approach the real information distribution, thus improving the accuracy of the target detection classification feature map through the classifier to obtain the classification result. Latent feature expression is a feature encoding method based on latent variables, which can construct a probability distribution function of latent variables according to the eigenvalues ​​of the target detection classification feature map and the posterior distribution of the target classification function, and is used to probabilistically encode and decode the eigenvalues ​​of the target detection classification feature map, so that it is more consistent with the posterior distribution of the target classification function, which can not only improve the accuracy and efficiency of feature encoding, but also maintain the original information and semantics of the features, avoiding feature information loss and confusion.

[0059] Finally, the optimized target detection classification feature map is passed through a classifier to obtain a classification result, and the classification result is used to indicate whether a target object exists. It should be understood that the optimized target detection classification feature map is obtained by processing, extracting and optimizing the comprehensive feature map of the vehicle environment through a deep learning network, and it contains feature representations of different positions in the image. By inputting the feature map into the classifier, the features of each position can be classified to determine whether the position belongs to the category of the target object. The classifier is usually a trained machine learning model that can classify the input features and output the corresponding category probability or confidence. By inputting the optimized target detection classification feature map into the classifier, the feature-category relationship learned by the model can be used to classify each position in the image. The classification result indicates whether there is information about the target object. Through the output of the classifier, the probability or confidence of each position belonging to the target object category can be obtained. According to the set threshold, the position with a probability higher than the threshold can be judged as the presence of the target object, and the position with a probability lower than the threshold can be judged as the absence of the target object. The significance of this step is to combine the optimized target detection classification feature map with the classifier to achieve classification determination of the target object in the image. Through the classifier, the target classification can be performed at each location to determine whether the target object exists in the image.

[0060] Specifically, the step of passing the optimized target detection classification feature map through a classifier to obtain a classification result, wherein the classification result is used to indicate whether a target object exists, includes: using the classifier to process the optimized target detection classification feature map using the following classification formula to generate the classification result; wherein the classification formula is:

[0061] O=softmax{(Wn , B n ):…:(W1,B1)|Project(F)}

[0062] Where O is the classification result, Project(F) represents the projection of the optimized target detection classification feature map into a vector, W1 to W n is the weight matrix of each fully connected layer, B1 to B n Represents the bias vector of each fully connected layer, and softmax is a normalized exponential function.

[0063] In summary, the scene target detection method based on deep learning in the embodiment of the present application is explained, which adopts artificial intelligence detection technology based on machine vision, obtains the vehicle surrounding environment image during the driving process of the vehicle through the on-board camera, and extracts the features of the vehicle surrounding environment image in high-dimensional space to analyze and determine whether there is a target object. In this way, real-time target detection can be achieved during the driving process, which helps to perceive and understand the road environment more accurately.

[0064] Figure 4 is a system block diagram of a scene object detection system based on deep learning according to an embodiment of the present application. Figure 4 As shown, according to the deep learning-based scene target detection system 100 of the embodiment of the present application, it includes: a vehicle surrounding environment data acquisition module 110, which is used to acquire a vehicle surrounding environment image taken by a vehicle-mounted camera; an environment image preprocessing module 120, which is used to perform image preprocessing on the vehicle surrounding environment image to obtain a preprocessed vehicle surrounding environment image; an environment feature extraction module 130, which is used to pass the preprocessed vehicle surrounding environment image through an environment feature extractor based on a hybrid convolutional layer to obtain a plurality of vehicle environment feature maps; an environment feature fusion module 140, which is used to fuse the plurality of vehicle environment feature maps to obtain a vehicle environment comprehensive feature map; a target detection feature generation module 150, which is used to input the vehicle environment comprehensive feature map into a Shuffle NetV2 basic module to obtain a target detection classification feature map; an optimization module 160, which is used to optimize the target detection classification feature map to obtain an optimized target detection classification feature map; a target detection result generation module 170, which is used to pass the optimized target detection classification feature map through a classifier to obtain a classification result, and the classification result is used to indicate whether a target object exists.

[0065] Here, those skilled in the art will appreciate that the specific functions and operations of the various units and modules in the above-mentioned scene object detection system based on deep learning 100 have been described in detail above. Figure 1 The description of the deep learning-based scene object detection method in FIG. 3 has been introduced in detail, and therefore, its repeated description will be omitted.

[0066] In summary, the scene target detection system 100 based on deep learning according to the embodiment of the present application is explained, which adopts artificial intelligence detection technology based on machine vision, obtains the vehicle surrounding environment image during the driving process of the vehicle through the vehicle-mounted camera, and extracts the features of the vehicle surrounding environment image in high-dimensional space to analyze and determine whether there is a target object. In this way, real-time target detection can be achieved during the driving process of the vehicle, which helps to perceive and understand the road environment more accurately.

[0067] As described above, the scene target detection system 100 based on deep learning according to the embodiment of the present application can be implemented in various wireless terminals, such as a server for scene target detection based on deep learning, etc. In one example, the scene target detection system 100 based on deep learning according to the embodiment of the present application can be integrated into the wireless terminal as a software module and / or a hardware module. For example, the scene target detection system 100 based on deep learning can be a software module in the operating system of the wireless terminal, or it can be an application developed for the wireless terminal; of course, the scene target detection system 100 based on deep learning can also be one of the many hardware modules of the wireless terminal.

[0068] Alternatively, in another example, the deep learning-based scene target detection system 100 and the wireless terminal may also be separate devices, and the deep learning-based scene target detection system 100 may be connected to the wireless terminal via a wired and / or wireless network and transmit interactive information in accordance with an agreed data format.

[0069] Below, reference Figure 5 To describe an electronic device according to an embodiment of the present application. Figure 5 FIG. 1 is a block diagram of an electronic device according to an embodiment of the present application. Figure 5 As shown, the electronic device may include: a processor 10, a communications interface 12, a memory 13 and a communication bus 14, wherein the processor 11, the communications interface 12 and the memory 13 communicate with each other through the communication bus 14. The processor 11 may call the logic instructions in the memory 13 to execute the above-mentioned network traffic anomaly detection method.

[0070] In addition, the logic instructions in the above-mentioned memory 13 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

Claims

1. A scene object detection method based on deep learning, characterized in that: include: Acquire an image of the vehicle's surroundings captured by a vehicle-mounted camera; Performing image preprocessing on the vehicle surrounding environment image to obtain a preprocessed vehicle surrounding environment image; Passing the preprocessed vehicle surrounding environment image through an environment feature extractor based on a hybrid convolutional layer to obtain a plurality of vehicle environment feature maps; fusing the plurality of vehicle environment characteristic maps to obtain a vehicle environment comprehensive characteristic map; Input the vehicle environment comprehensive feature map into the Shuffle NetV2 basic module to obtain a target detection classification feature map; Optimizing the target detection classification feature map to obtain an optimized target detection classification feature map; Passing the optimized target detection classification feature map through a classifier to obtain a classification result, wherein the classification result is used to indicate whether a target object exists; The step of optimizing the target detection classification feature map to obtain an optimized target detection classification feature map comprises: calculating the latent feature expression of the motion parameterization model of the target detection classification feature map relative to the target classification function using the following optimization formula to obtain the optimized target detection classification feature map; Wherein, the optimization formula is: Among them, f i,j,k represents the feature value of the (i, j, k)th position of the target detection classification feature map, log represents the logarithmic function value with base 2, λ represents a predetermined hyperparameter, and f i,j,k ' represents the feature value of the (i, j, k)th position of the optimized target detection classification feature map.

2. The scene object detection method based on deep learning according to claim 1, characterized in that: The hybrid convolution layer includes a parallel first convolution branch structure, a second convolution branch structure, a third convolution branch structure and a fourth convolution branch structure, wherein the first convolution branch uses a first convolution kernel having a first size, the second convolution branch uses a second convolution kernel having a first size and a first dilation rate, the third convolution branch uses a third convolution kernel having a first size and a second dilation rate, and the fourth convolution branch uses a fourth convolution kernel having a first size and a third dilation rate.

3. The scene object detection method based on deep learning according to claim 2, characterized in that: The preprocessed vehicle surrounding environment image is passed through an environment feature extractor based on a hybrid convolutional layer to obtain a plurality of vehicle environment feature maps, including: Using the first convolution kernel with the first size to perform convolution encoding on the preprocessed vehicle surrounding environment image to obtain a first-scale vehicle environment feature map; Using the second convolution kernel with the first dilation rate to perform convolution encoding on the preprocessed vehicle surrounding environment image to obtain a second-scale vehicle environment feature map; Using the third convolution kernel with the second dilation rate to perform convolution encoding on the preprocessed vehicle surrounding environment image to obtain a third-scale vehicle environment feature map; The preprocessed vehicle surrounding environment image is convolutionally encoded using the fourth convolution kernel with the third dilation rate to obtain a fourth-scale vehicle environment feature map.

4. The scene object detection method based on deep learning according to claim 3, characterized in that: Passing the optimized target detection classification feature map through a classifier to obtain a classification result, wherein the classification result is used to indicate whether a target object exists, including: using the classifier to process the optimized target detection classification feature map using the following classification formula to generate the classification result; Among them, the classification formula is: O=softmax{(W n ,B n ):...:(W1,B1)|Project(F)} Where O is the classification result, Project(F) represents the projection of the optimized target detection classification feature map into a vector, W1 to W n is the weight matrix of each fully connected layer, B1 to B n Represents the bias vector of each fully connected layer, and softmax is a normalized exponential function.

5. A scene object detection system based on deep learning, characterized in that: include: A vehicle surrounding environment data acquisition module is used to acquire the vehicle surrounding environment image taken by the vehicle-mounted camera; An environment image preprocessing module, used for performing image preprocessing on the vehicle surrounding environment image to obtain a preprocessed vehicle surrounding environment image; An environmental feature extraction module, used for passing the pre-processed vehicle surrounding environment image through an environmental feature extractor based on a hybrid convolutional layer to obtain a plurality of vehicle environment feature maps; An environmental feature fusion module, used for fusing the multiple vehicle environmental feature maps to obtain a comprehensive vehicle environmental feature map; A target detection feature generation module, used for inputting the vehicle environment comprehensive feature map into the Shuffle NetV2 basic module to obtain a target detection classification feature map; An optimization module, used for optimizing the target detection classification feature map to obtain an optimized target detection classification feature map; A target detection result generation module, used for passing the optimized target detection classification feature map through a classifier to obtain a classification result, wherein the classification result is used to indicate whether a target object exists; The optimization module is used to calculate the latent feature expression of the motion parameterization model of the target detection classification feature map relative to the target classification function using the following optimization formula to obtain the optimized target detection classification feature map; Wherein, the optimization formula is: Among them, f i,j,k represents the feature value of the (i, j, k)th position of the target detection classification feature map, log represents the logarithmic function value with base 2, λ represents a predetermined hyperparameter, and f i,j,k ' represents the feature value of the (i, j, k)th position of the optimized target detection classification feature map.

6. The scene object detection system based on deep learning according to claim 5, characterized in that: The environmental feature extraction module comprises: A first-scale vehicle environment feature extraction unit, configured to perform convolution encoding on the preprocessed vehicle surrounding environment image using a first convolution kernel having a first size to obtain a first-scale vehicle environment feature map; A second-scale vehicle environment feature extraction unit, configured to perform convolution encoding on the preprocessed vehicle surrounding environment image using a second convolution kernel having a first void ratio to obtain a second-scale vehicle environment feature map; A third-scale vehicle environment feature extraction unit, configured to perform convolution encoding on the preprocessed vehicle surrounding environment image using a third convolution kernel having a second hole rate to obtain a third-scale vehicle environment feature map; The fourth-scale vehicle environment feature extraction unit is used to perform convolution encoding on the preprocessed vehicle surrounding environment image using a fourth convolution kernel with a third void rate to obtain a fourth-scale vehicle environment feature map.

7. The scene object detection system based on deep learning according to claim 6, characterized in that: The target detection result generating module includes: using the classifier to process the optimized target detection classification feature map with the following classification formula to generate the classification result; Among them, the classification formula is: O=softmax{(W n ,B n ):...:(W1,B1)|Project(F)} Where O is the classification result, Project(F) represents the projection of the optimized target detection classification feature map into a vector, W1 to W n is the weight matrix of each fully connected layer, B1 to B n Represents the bias vector of each fully connected layer, and softmax is a normalized exponential function.

8. An electronic device comprising a memory and a processor coupled to the memory, characterized in that: The processor is configured to execute the scene object detection method based on deep learning as described in any one of claims 1 to 4 based on the instructions stored in the memory.

Images