Obstacle detection method, device, system and storage medium

Abstract

The application provides an obstacle detection method, device, system and storage medium. The method comprises: acquiring a detection picture collected by a sensor; and inputting the detection picture into a preset improved yolov5 model for processing, wherein the improved yolov5 model comprises a backbone network, a neck network and a head network; wherein the improved yolov5 model has an improved csp structure in the backbone network and the neck network, the improved csp structure comprises a first processing branch, a second processing branch and a main part, the first processing branch comprises a first calculation module, the second processing branch comprises a first calculation module and a second calculation module, and the main part comprises a fusion layer and a first calculation module; wherein when the second calculation module processes a feature map, each feature map is independently processed by a convolution kernel larger than a conventional convolution kernel, and a channel of each feature map processed by convolution is processed by a conventional convolution operation and a weighting operation. The method improves the precision and the calculation speed.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned driving, and particularly relates to an obstacle detection method, device and system and a storage medium. BACKGROUND

[0002] In the implementation process of the automatic obstacle avoidance function in the unmanned driving technology, obstacle detection is a very important link. The deep learning algorithm model yolov5 model has sufficient detection accuracy and speed, and is widely used in obstacle detection.

[0003] In the prior art, in order to meet the real-time requirement of the yolov5 model, a lightweight classification network is generally combined with the yolov5 model to improve the model detection speed.

[0004] However, the inventors have found that the prior art at least has the following technical problems: replacing the backbone network in the yolov5 model with a lightweight classification network reduces the feature extraction capability of the model and reduces the model detection accuracy. SUMMARY

[0005] The present application provides an obstacle detection method, device, system and storage medium to overcome the problem of low model detection accuracy after improving the yolov5 model with a lightweight classification network in the prior art.

[0006] In a first aspect, the present application provides an obstacle detection method, comprising:

[0007] obtaining a detection picture collected by a sensor;

[0008] inputting the detection picture into a preset improved yolov5 model for processing, wherein the improved yolov5 model comprises a backbone network, a neck network and a head network, and the specific processing process of the improved yolov5 model comprises: inputting the detection picture into the backbone network for first feature extraction processing, inputting the feature map after the first feature extraction processing into the neck network for second feature extraction processing, and inputting the feature map after the second feature extraction processing into the head network to output an obstacle detection result;

[0009] The improved csp structure includes a first processing branch, a second processing branch and a trunk part, the first processing branch includes a first calculation module, the second processing branch includes a first calculation module and a second calculation module, and the trunk part includes a fusion layer and a first calculation module; wherein the second calculation module includes a depth separable convolution with a size larger than a regular convolution kernel; wherein the processing process of the improved csp structure on the feature map is: inputting the feature map into the first processing branch, processing it through the first calculation module; inputting the feature map into the second processing branch, processing it through the first calculation module and the second calculation module in turn; inputting the feature map processed by the first processing branch and the feature map processed by the second processing branch into the fusion layer of the trunk part for fusion to obtain a fused feature map; processing the fused feature map through the first calculation module of the trunk part to obtain a feature map processed by the improved csp structure, wherein when the second calculation module processes the feature map, the depth separable convolution with a size larger than a regular convolution kernel is used to process the feature map as follows: each feature map is independently convolved through the depth separable convolution with a size larger than a regular convolution kernel, and a regular convolution operation and a weighting processing are performed on the channels of the feature maps processed by convolution.

[0010] In a possible design, the first calculation module includes, in sequence from input to output, a first regular convolution layer, a first batch normalization layer and a first activation function layer; accordingly, inputting the feature map into the first processing branch for processing through the first calculation module to output a first feature map includes: performing convolution processing on the feature map through the first regular convolution layer to obtain a feature map with a halved number; performing batch normalization processing on the feature map with a halved number through the first batch normalization layer to obtain a feature map subject to a standard normal distribution; and increasing the nonlinear characteristics of the feature map subject to a standard normal distribution through the first activation function layer to output the feature map processed by the first processing branch.

[0011] In a possible design, the second computing module sequentially comprises, in the order from input to output, a second batch normalization layer, a second regular convolution layer, a third batch normalization layer, a second activation function layer, a depth separable convolution with a size larger than a regular convolution kernel, a fourth batch normalization layer, a third activation function layer, a third regular convolution layer, and a fifth batch normalization layer; the depth separable convolution with the size larger than the regular convolution kernel comprises a plurality of depth-wise convolutions and a point-wise convolution; accordingly, the depth separable convolution with the size larger than the regular convolution kernel is performed on each feature map independently, and a regular convolution operation and a weighting process are performed on channels of the feature maps after the convolution; the depth-wise convolutions are used to perform independent convolution calculation on each feature map to obtain the feature maps after the convolution; and the point-wise convolution is used to perform the regular convolution operation on all the feature maps after the convolution, and the weighting process is performed on the channels obtained through the regular convolution operation to obtain the feature maps after the channel weighting combination.

[0012] In a possible design, the second computing module further comprises a residual layer; the residual layer is configured to perform residual connection processing on a feature map input into the second batch normalization layer and a feature map output from the fifth batch normalization layer to obtain a feature map after the residual connection.

[0013] In a possible design, the depth-wise convolution uses a convolution kernel with a size of 7*7.

[0014] In a second aspect, the present application provides an obstacle detection device, comprising:

[0015] An acquisition module is configured to acquire a detection picture collected by a sensor.

[0016] A processing module is configured to input the detection picture into a preset improved yolov5 model for processing, wherein the improved yolov5 model comprises a backbone network, a neck network, and a head network; the processing module specifically comprises: a first extraction unit, configured to input the detection picture into the backbone network for first feature extraction processing; a second extraction unit, configured to input a feature map after the first feature extraction processing into the neck network for second feature extraction processing; and an output unit, configured to input a feature map after the second feature extraction processing into the head network to output an obstacle detection result.

[0017] The improved csp structure includes a first processing branch, a second processing branch, and a trunk part, the first processing branch includes a first calculation module, the second processing branch includes a first calculation module and a second calculation module, and the trunk part includes a fusion layer and a first calculation module; the second calculation module includes a depth separable convolution with a size larger than a regular convolution kernel; the improved csp structure includes a first input subunit, a second input subunit, a fusion subunit, and a third input subunit; the first input subunit is specifically configured to input a feature map into the first processing branch and process the feature map through the first calculation module; the second input subunit is specifically configured to input the feature map into the second processing branch and process the feature map through the first calculation module and the second calculation module in sequence; the fusion subunit is specifically configured to input the feature map processed by the first processing branch and the feature map processed by the second processing branch into the fusion layer of the trunk part for fusion to obtain a fused feature map; and the first input subunit is specifically configured to process the fused feature map through the first calculation module of the trunk part to obtain a feature map processed by the improved csp structure; when the second calculation module processes the feature map, the depth separable convolution with the size larger than the regular convolution kernel is used to process the feature map as follows: each feature map is independently convolved, and a regular convolution operation and a weighting process are performed on the channels of the feature maps processed by convolution.

[0018] In a possible design, the first input subunit is specifically configured to perform convolution processing on the feature map through a first regular convolution layer to obtain a feature map with a halved number; perform batch normalization processing on the feature map with the halved number through a first batch normalization layer to obtain a feature map subject to a standard normal distribution; and increase the nonlinear characteristics of the feature map subject to the standard normal distribution through a first activation function layer to output a feature map processed by the first processing branch.

[0019] In a possible design, the second calculation module includes, in sequence from input to output, a second batch normalization layer, a second regular convolution layer, a third batch normalization layer, a second activation function layer, a depth separable convolution with a size larger than a regular convolution kernel, a fourth batch normalization layer, a third activation function layer, a third regular convolution layer, and a fifth batch normalization layer; the depth separable convolution with the size larger than the regular convolution kernel includes a plurality of depth-wise convolutions and a point-wise convolution; and correspondingly, the second input subunit is specifically configured to perform independent convolution calculation on each feature map through each depth-wise convolution layer to obtain feature maps processed by convolution; and perform convolution processing on all the feature maps processed by convolution through the point-wise convolution to obtain a feature map after channel weighting combination.

[0020] In a third aspect, the present application provides an obstacle detection system, comprising:

[0021] an image sensor configured to acquire a detection image;

[0022] an electronic control unit comprising at least one processor and a memory;

[0023] the memory stores computer-executable instructions;

[0024] the at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the obstacle detection method according to the first aspect and various possible designs of the first aspect.

[0025] In a fourth aspect, the present application provides a computer storage medium, wherein the computer storage medium stores computer-executable instructions, and when the processor executes the computer-executable instructions, the obstacle detection method according to the first aspect and various possible designs of the first aspect is implemented.

[0026] In a fifth aspect, the present application provides a computer program product, comprising a computer program, and when the processor executes the computer program, the obstacle detection method according to the first aspect and various possible designs of the first aspect is implemented.

[0027] The obstacle detection method, device, system and storage medium provided by the present application introduce an improved csp structure in the yolov5 model, increase the receptive field of obstacle detection by using a depth separable convolution with a size larger than a conventional convolution kernel in the improved csp structure, retain more global information of the obstacle, and improve the accuracy; at the same time, the depth separable convolution is used to replace the conventional convolution to construct a large kernel convolution, thereby reducing the model calculation amount and improving the calculation speed. BRIEF DESCRIPTION OF DRAWINGS

[0028] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0029] Figure 1 The application scenario diagram of the obstacle detection method provided by the present application is shown.

[0030] Figure 2 The structure diagram of the improved yolov5 model provided by an embodiment of the present application is shown.

[0031] Figure 3 The improved csp structure diagram provided by an embodiment of the present application is shown.

[0032] Figure 4 The structural diagram of the depth separable convolution provided for an embodiment of the present application is shown in the obstacle detection method flowchart.

[0033] Figure 5 The specific calculation process diagram of the depth-wise convolution provided for an embodiment of the present application is shown in the obstacle detection method flowchart.

[0034] Figure 6 The structural diagram of the obstacle detection device provided for an embodiment of the present application is shown in the obstacle detection method flowchart.

[0035] Figure 7 The hardware structure diagram of the system provided for an embodiment of the present application is shown in the obstacle detection method flowchart. DETAILED DESCRIPTION

[0036] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below in connection with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some, but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0037] First, the terms involved in the embodiments of the present application are explained:

[0038] 1. The yolov5 model: the yolov5 model is a single-stage target detection model, which adds some improved ideas on the basis of the yolov4 model, so that the speed and accuracy are greatly improved.

[0039] 2. The csp structure: is a part of the yolov5 structure, which is used to divide the original input into two branches, and perform convolution operation respectively to reduce the channel number by half, so that the model learns more features.

[0040] In view of the problem that the model detection accuracy is low after the yolov5 model is improved by using a lightweight classification network in the prior art, the present disclosure provides the following technical concept: the csp structure in the yolov5 model is improved, and a large-core convolution is introduced in the csp structure to increase the effective receptive field and reduce the loss of global features of obstacles, thereby improving the accuracy of model detection.

[0041] Figure 1 The application scenario diagram of the obstacle detection method provided for an embodiment of the present application is shown in the obstacle detection method flowchart. Figure 1As shown, it includes an image sensor 101 and an electronic control device 102. Among them, the image sensor 101 sends the detection picture collected at the current moment to the electronic control device 102, and the electronic control device 102 analyzes the obtained detection picture to obtain the obstacle detection result.

[0042] The technical solutions of the present application and how the technical solutions of the present application solve the above technical problems will be described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of the present application will be described below with reference to the drawings.

[0043] The execution subject of the present embodiment can be Figure 1 The electronic control device in the embodiment shown can also be other forms of computer devices, and the present embodiment does not make special restrictions here. The method comprises:

[0044] S201: Obtain the detection picture collected by the sensor.

[0045] In the present embodiment, the sensor can be an image sensor of the vehicle body.

[0046] Specifically, the detection picture of the environment around the vehicle collected by the image sensor is received.

[0047] S202: Input the detection picture into a preset improved yolov5 model for processing, wherein the improved yolov5 model comprises a backbone network, a neck network and a head network, wherein the backbone network and the neck network have an improved csp structure, wherein the improved csp structure comprises a first processing branch, a second processing branch and a main part, the first processing branch comprises a first calculation module, the second processing branch comprises a first calculation module and a second calculation module, and the main part comprises a fusion layer and a first calculation module; wherein the second calculation module comprises a depth separable convolution with a size larger than a conventional convolution kernel.

[0048] Figure 2 The structure diagram of the improved yolov5 model, and the specific processing process comprises:

[0049] S2021: Input the detection picture into the backbone network for first feature extraction processing, and input the feature map after the first feature extraction processing into the neck network for second feature extraction processing, wherein the improved csp structure in the backbone network and the neck network is as shown in Figure 3 The specific processing process comprises:

[0050] Sa: Input the feature map into the first processing branch and process it through the first calculation module.

[0051] The first calculation module sequentially comprises, in order from input to output, a first regular convolutional layer, a first batch normalization layer, and a first activation function layer.

[0052] Specifically, the feature map is subjected to convolutional processing by the first regular convolutional layer to obtain a feature map with a number halved, the feature map with the number halved is subjected to batch normalization processing by the first batch normalization layer to obtain a feature map subject to standard normal distribution, and the feature map subject to standard normal distribution is subjected to nonlinear processing by the first activation function layer to obtain a feature map processed by the first processing branch.

[0053] In this embodiment, the activation function can be a LeakyReLU activation function.

[0054]

[0055] In the formula, x is an independent variable of the activation function, and the value of a is usually about 0.01.

[0056] Sb: inputting the feature map into the second processing branch and sequentially processing the feature map by the first calculation module and the second calculation module; wherein when processing the feature map, the second calculation module subjects the feature map to processing by the depthwise separable convolution with a size exceeding a regular convolutional kernel as follows: independently processing each feature map by the depthwise separable convolution with a size exceeding a regular convolutional kernel, and performing regular convolutional operation and weighting processing on the channels of the feature maps processed by convolution.

[0057] In this embodiment, the second calculation module sequentially comprises, in order from input to output, a second batch normalization layer, a second regular convolutional layer, a third batch normalization layer, a second activation function layer, a depthwise separable convolution with a size exceeding a regular convolutional kernel, a fourth batch normalization layer, a third activation function layer, a third regular convolutional layer, and a fifth batch normalization layer.

[0058] The depthwise separable convolution with a size exceeding a regular convolutional kernel comprises a plurality of depthwise convolutions and a pointwise convolution.

[0059] Specifically, the independent convolutional calculation on each feature map by each depthwise convolution layer obtains the feature maps processed by convolution, and the pointwise convolution performs regular convolutional operation on all the feature maps processed by convolution and weighting processing on the channels obtained by the regular convolutional operation to obtain the feature map after channel weighting combination.

[0060] The independent convolutional calculation on each feature map by each depthwise convolution layer obtains the feature maps processed by convolution, and the pointwise convolution performs regular convolutional operation on all the feature maps processed by convolution and weighting processing on the channels obtained by the regular convolutional operation to obtain the feature map after channel weighting combination.

[0061] For example, a 7*7 convolutional kernel is used in the depthwise convolution.

[0062] Specifically, a structural diagram of the depth separable convolution is shown in Figure 4 As shown in the figure, it is divided into two-step operations of depth-wise convolution and point-wise convolution. Each 7*7 size depth-wise convolution performs point-to-point multiplication and addition calculation on the pixel data of each feature map to obtain the feature map after convolution processing. Then the pixel data of the feature map after convolution processing is input into the point-wise convolution for 1*1 convolution operation, and the feature map after convolution processing is weighted and combined to obtain the feature map after channel weighted combination.

[0063] The specific calculation process diagram of the depth-wise convolution is shown in Figure 5 As shown in the figure, the convolution kernel performs point-to-point multiplication and addition calculation on each pixel data, and the specific calculation formula is as follows:

[0064]

[0065] In the formula, y i is the i-th pixel data of the output feature map, W j is the j-th element in the convolution kernel, and X j is the j-th pixel data in the corresponding receptive field in the input feature map.

[0066] Sc: input the feature map processed by the first processing branch and the feature map processed by the second processing branch into the fusion layer of the backbone part for fusion to obtain the fused feature map.

[0067] Specifically, the feature map processed by the first processing branch and the feature map processed by the second processing branch are input into the fusion layer of the backbone part for fusion.

[0068] In this embodiment, the fusion layer can use the concat function, which is a function that increases the number of channels.

[0069] Sd: processing the fused feature map through the first calculation module of the backbone part to obtain the feature map processed by the improved csp structure.

[0070] S2022: input the feature map processed by the second feature extraction into the head network to output the obstacle detection result.

[0071] In summary, the obstacle detection method provided in this embodiment introduces an improved csp structure in the yolov5 model, wherein the improved csp structure contains a depth separable convolution with a size larger than a conventional convolution kernel, which increases the receptive field of obstacle detection, retains more global information of the obstacle, and improves the accuracy; at the same time, the depth separable convolution is used to replace the conventional convolution to construct a large kernel convolution, which reduces the model calculation amount and improves the calculation speed.

[0072] In addition, the feature map is input into the first convolutional layer for processing, so that the number of channels is halved, and the calculation speed of the model is accelerated; the feature map with the number halved is subjected to batch normalization processing, so that the pixel data of the feature map is subject to standard normal distribution, and the stability of the model is enhanced; the feature map subject to the standard normal distribution is input into the activation function, and the non-linear characteristics of the model are increased.

[0073] In addition, the calculation speed of the model is improved by the depth separable convolution, while the accuracy of the model is ensured, wherein the point-by-point convolution in the depth separable convolution calculates only one corresponding feature map for each convolution kernel, effectively reducing the calculation amount of the model, and the weighting combination between the channels of the feature map is performed after the point-by-point convolution, effectively utilizing the feature map pixel data at the same position in different feature maps, and ensuring the accuracy of the model detection.

[0074] The embodiment is based on Figure 2 On the basis of the embodiment, the residual layer structure arranged in the second calculation module is introduced. The residual layer is also included in the second calculation module. The residual layer is used to perform residual connection processing on the feature map input into the second batch normalization layer and the feature map output by the fifth batch normalization layer in the second calculation module, to obtain the feature map after residual connection.

[0075] Specifically, the calculation formula of the pixel data residual connection in the feature map is as follows:

[0076] y = x + f(x)

[0077] In the formula, y is the pixel data of the feature map output by the second calculation module Res Unit, x is the pixel data of the feature map input into the second batch normalization layer, and f(x) is the pixel data of the feature map output by the fifth batch normalization layer in the second calculation module Res Unit.

[0078] In the embodiment, the residual layer can use the add function, which is a function of adding the number of feature maps without changing the number of channels.

[0079] In summary, the obstacle detection method provided in the embodiment adds the feature map input into the second batch normalization layer to the feature map output by the fifth batch normalization layer in the second calculation module, performs residual connection on the feature map output by the fifth batch normalization layer in the second calculation module, makes up for the edge information of the feature map destroyed during convolution due to the too large size of the convolution kernel, and increases the detection accuracy of the edge obstacle.

[0080] Figure 6 The structure diagram of the obstacle detection device provided in the embodiment of the application is shown in FIG. 6. Figure 6 As shown in the figure, the obstacle detection device includes an acquisition module 601 and a processing module 602.

[0081] The acquisition module 601 is configured to acquire a detection picture collected by a sensor.

[0082] The processing module 602 is configured to input the detection picture into a preset improved yolov5 model for processing, wherein the improved yolov5 model comprises a backbone network, a neck network and a head network, and the processing module specifically comprises: a first extraction unit, specifically configured to input the detection picture into the backbone network for first feature extraction processing; a second extraction unit, specifically configured to input a feature map after the first feature extraction processing into the neck network for second feature extraction processing; and an output unit, specifically configured to input a feature map after the second feature extraction processing into the head network to output an obstacle detection result.

[0083] The improved csp structure is included in the first extraction unit and the second extraction unit, wherein the improved csp structure comprises a first processing branch, a second processing branch and a main part, the first processing branch comprises a first calculation module, the second processing branch comprises a first calculation module and a second calculation module, and the main part comprises a fusion layer and a first calculation module; the second calculation module comprises a depth separable convolution with a size larger than a conventional convolution kernel; the improved csp structure comprises a first input subunit, a second input subunit, a fusion subunit and a third input subunit; the first input subunit is specifically configured to input the feature map into the first processing branch and perform processing through the first calculation module; the second input subunit is specifically configured to input the feature map into the second processing branch and perform processing through the first calculation module and the second calculation module in sequence; the fusion subunit is specifically configured to input the feature map processed by the first processing branch and the feature map processed by the second processing branch into the fusion layer of the main part for fusion to obtain a fused feature map; and the first input subunit is specifically configured to perform processing on the fused feature map through the first calculation module of the main part to obtain a feature map processed by the improved csp structure.

[0084] When the second calculation module processes the feature map, the depth separable convolution with a size larger than a conventional convolution kernel is used to process the feature map as follows: each feature map is independently convolved through a conventional convolution kernel with a size larger than a conventional convolution kernel, and a conventional convolution operation and weighting processing are performed on the channels of the feature maps after convolution processing.

[0085] In a possible design, the first input subunit is specifically configured to: perform convolution processing on the feature map through a first conventional convolution layer to obtain a feature map with a halved number of channels; perform batch normalization processing on the feature map with the halved number of channels through a first batch normalization layer to obtain a feature map subject to a standard normal distribution; and increase the nonlinear characteristics of the feature map subject to the standard normal distribution through a first activation function layer to output a feature map processed by the first processing branch.

[0086] In a possible design, the second calculation module sequentially comprises, in the order from input to output, a second batch normalization layer, a second regular convolution layer, a third batch normalization layer, a second activation function layer, a depth separable convolution with a size larger than a regular convolution kernel, a fourth batch normalization layer, a third activation function layer, a third regular convolution layer, and a fifth batch normalization layer; the depth separable convolution with the size larger than the regular convolution kernel comprises a plurality of depth-wise convolutions and a point-wise convolution; and the second input subunit is specifically configured to: obtain feature maps processed by the convolutions by performing independent convolution calculation on each feature map in each depth-wise convolution layer; and obtain a feature map combined in a channel by weight by performing convolution processing on all the feature maps processed by the convolutions.

[0087] In a possible design, the second calculation module further comprises a residual layer; the residual layer is configured to perform residual connection processing on a feature map input to the second batch normalization layer and a feature map output by the fifth batch normalization layer, to obtain a feature map after residual connection.

[0088] In a possible design, the depth-wise convolution uses a convolution kernel with a size of 7*7.

[0089] The apparatus provided in this embodiment can be used to execute the technical solutions of the method embodiments, and has similar implementation principles and technical effects, which will not be described here again in this embodiment.

[0090] The obstacle detection system in this embodiment comprises:

[0091] The image sensor 101 is configured to collect a detection picture.

[0092] The electronic control unit 102 comprises at least one processor 701 and a memory 702. Figure 7 The structure diagram of the electronic control unit provided in this embodiment comprises at least one processor 701 and a memory 702.

[0093] The memory 702 is configured to store computer execution instructions.

[0094] The processor 701 is configured to execute the computer execution instructions stored in the memory, to implement each step performed by the electronic control device in the above-described embodiments. For details, refer to the related description in the foregoing method embodiments.

[0095] Optionally, the memory 702 can be independent or integrated with the processor 701.

[0096] When the memory 702 is independently arranged, the electronic control device further comprises a bus 703 configured to connect the memory 702 and the processor 701.

[0097] The embodiment of the present application further provides a computer storage medium, and the computer storage medium stores computer execution instructions. When a processor executes the computer execution instructions, the obstacle detection method is realized.

[0098] The embodiment of the present application further provides a computer program product, and the computer program product comprises a computer program. When the computer program is executed by a processor, the obstacle detection method is realized. The embodiment of the present application further provides a computer program product, and the computer program product comprises a computer program. When the computer program is executed by a processor, the obstacle detection method is realized.

[0099] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other manners. For example, the above-described device embodiments are merely illustrative. For example, the division of the modules is merely logical function division. In actual implementation, another division manner can be used. For example, a plurality of modules can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed coupling or direct coupling or communication connection between modules can be indirect coupling or communication connection through some interfaces, devices or modules, and can be electrical, mechanical or in other forms.

[0100] The modules illustrated as separated components can or can not be physically separated, and the components illustrated as modules can or can not be physical units, that is, can be located in one place, or can be distributed on a plurality of network units. Part or all of the modules can be selected according to actual needs to implement the embodiment scheme.

[0101] In addition, each function module in each embodiment of the present application can be integrated in one processing unit, or each module can be physically present alone, or two or more modules can be integrated in one unit. The unit formed by the above modules can be realized in the form of hardware, or in the form of hardware plus software function unit.

[0102] The integrated module realized in the form of software function module can be stored in a computer readable storage medium. The software function module stored in the storage medium comprises a plurality of instructions for enabling a computer device (which can be a personal computer, an electronic control device or a network device, etc.) or a processor to execute part of steps of the method of each embodiment of the present application.

[0103] ​​​​​​It should be appreciated that the above processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in combination with the application can be directly embodied as hardware processor execution, or executed by a combination of hardware and software modules in the processor.

[0104] The memory can include a high-speed RAM memory, and can also include a non-volatile storage NVM, such as at least one disk memory, and can also be a U disk, a mobile hard disk, a read-only memory, a magnetic disk or an optical disk, etc.

[0105] The bus can be an industry standard architecture (ISA) bus, a peripheral component interconnect (PCI) bus, or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of representation, the bus in the drawings of the present application does not limit to only one bus or one type of bus.

[0106] The above storage medium can be realized by any type of volatile or non-volatile storage device or their combination, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk. The storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0107] An exemplary storage medium is coupled to the processor, so that the processor can read information from the storage medium and can write information to the storage medium. Of course, the storage medium can also be an integral part of the processor. The processor and the storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the storage medium can also exist as discrete components in an electronic control device or a host device.

[0108] Those skilled in the art can understand that all or part of the steps of the above-mentioned method embodiments can be completed by program instruction related hardware. The foregoing program can be stored in a computer readable storage medium. The program executes to perform the steps of the above-mentioned method embodiments; and the foregoing storage medium includes various storage media that can store program codes, such as ROM, RAM, magnetic disk or optical disk.

[0109] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. An obstacle detection method characterized by, The method comprises the following steps: acquiring a detection picture collected by a sensor; inputting the detection picture into a preset improved yolov5 model for processing, wherein the improved yolov5 model comprises a backbone network, a neck network and a head network, and the specific processing process of the improved yolov5 model comprises: inputting the detection picture into the backbone network for first feature extraction processing, inputting the feature map after the first feature extraction processing into the neck network for second feature extraction processing, and inputting the feature map after the second feature extraction processing into the head network to output an obstacle detection result; wherein the backbone network and the neck network have an improved csp structure, wherein the improved csp structure comprises a first processing branch, a second processing branch and a main part, the first processing branch comprises a first calculation module, the second processing branch comprises a first calculation module and a second calculation module, and the main part comprises a fusion layer and a first calculation module; wherein the second calculation module comprises a depth separable convolution with a size larger than a conventional convolution kernel; wherein the processing process of the improved csp structure on the feature map is: inputting the feature map into the first processing branch, processing it through the first calculation module; inputting the feature map into the second processing branch, processing it through the first calculation module and the second calculation module in turn; inputting the feature map processed by the first processing branch and the feature map processed by the second processing branch into the fusion layer of the main part for fusion to obtain a fused feature map; processing the fused feature map through the first calculation module of the main part to obtain a feature map processed by the improved csp structure, wherein when the second calculation module processes the feature map, the depth separable convolution with a size larger than a conventional convolution kernel is used to process the feature map as follows: the depth separable convolution with a size larger than a conventional convolution kernel is used to independently convolve each feature map, and a conventional convolution operation and a weighting process are performed on the channels of the feature maps after the convolution.

2. The method of claim 1, wherein, The first calculation module comprises, in order from input to output, a first conventional convolution layer, a first batch normalization layer and a first activation function layer; Correspondingly, the processing of inputting the feature map into the first processing branch and processing it through the first calculation module comprises: convolving the feature map through the first conventional convolution layer to obtain a feature map with a number reduced by half; performing batch normalization on the feature map after the number reduction through the first batch normalization layer to obtain a feature map subject to standard normal distribution; increasing the nonlinearity of the feature map subject to standard normal distribution through the first activation function layer to output the feature map processed by the first processing branch.

3. The method of claim 1, wherein, The second calculation module comprises, in order from input to output, a second batch normalization layer, a second conventional convolution layer, a third batch normalization layer, a second activation function layer, a depth separable convolution with a size larger than a conventional convolution kernel, a fourth batch normalization layer, a third activation function layer, a third conventional convolution layer and a fifth batch normalization layer; wherein the depth separable convolution with a size larger than a conventional convolution kernel comprises a plurality of depth-wise convolutions and a point-wise convolution. Correspondingly, the independent convolution processing of each feature map by the size exceeding the conventional convolution kernel, and the conventional convolution operation and weighting processing on the channels of the feature maps after the convolution processing, include: obtaining the feature maps after the convolution processing by the independent convolution calculation of each feature map in each depth-wise convolution layer; performing the conventional convolution operation on all the feature maps after the convolution processing by the point-wise convolution, and performing the weighting processing on the channels obtained by the conventional convolution operation to obtain the feature maps after the channel weighting combination.

4. The method of claim 3, wherein, The second calculation module further includes a residual layer. The residual layer is used for residual connection processing of the feature maps input into the second batch normalization layer and the feature maps output by the fifth batch normalization layer to obtain the feature maps after the residual connection.

5. The method of claim 3, wherein, The depth-wise convolution uses a 7*7 size convolution kernel.

6. An obstacle detection device characterized by comprising: It includes: an acquisition module, configured to acquire a detection picture collected by a sensor; a processing module, configured to input the detection picture into a preset improved yolov5 model for processing, wherein the improved yolov5 model includes a backbone network, a neck network and a head network, and the processing module specifically includes: a first extraction unit, specifically configured to input the detection picture into the backbone network for first feature extraction processing; a second extraction unit, specifically configured to input the feature maps after the first feature extraction processing into the neck network for second feature extraction processing; and an output unit, specifically configured to input the feature maps after the second feature extraction processing into the head network to output an obstacle detection result. The improved csp structure includes a first processing branch, a second processing branch, and a trunk part, the first processing branch includes a first calculation module, the second processing branch includes a first calculation module and a second calculation module, and the trunk part includes a fusion layer and a first calculation module; the second calculation module includes a depth separable convolution with a size larger than a conventional convolution kernel; the improved csp structure includes a first input subunit, a second input subunit, a fusion subunit, and a third input subunit; the first input subunit is specifically configured to input a feature map into the first processing branch and process the feature map through the first calculation module; the second input subunit is specifically configured to input the feature map into the second processing branch and process the feature map through the first calculation module and the second calculation module in sequence; the fusion subunit is specifically configured to input the feature map processed by the first processing branch and the feature map processed by the second processing branch into the fusion layer of the trunk part for fusion to obtain a fused feature map; and the third input subunit is specifically configured to process the fused feature map through the first calculation module of the trunk part to obtain a feature map processed by the improved csp structure; when the second calculation module processes the feature map, the depth separable convolution with the size larger than the conventional convolution kernel is used to process the feature map as follows: each feature map is independently convolved, and a conventional convolution operation and weighting processing are performed on the channels of the feature maps processed by convolution.

7. The apparatus of claim 6, wherein, The first input subunit is specifically configured to perform convolution processing on the feature map through a first conventional convolution layer to obtain a feature map with a halved number; perform batch normalization processing on the feature map with the halved number through a first batch normalization layer to obtain a feature map subject to a standard normal distribution; and increase the nonlinear characteristics of the feature map subject to the standard normal distribution through a first activation function layer to output a feature map processed by the first processing branch.

8. The apparatus of claim 6, wherein, The second calculation module includes, in sequence from input to output, a second batch normalization layer, a second conventional convolution layer, a third batch normalization layer, a second activation function layer, a depth separable convolution with a size larger than a conventional convolution kernel, a fourth batch normalization layer, a third activation function layer, a third conventional convolution layer, and a fifth batch normalization layer; the depth separable convolution with the size larger than the conventional convolution kernel includes a plurality of depth-wise convolutions and a point-wise convolution; accordingly, the second input subunit is specifically configured to perform independent convolution calculation on each feature map through each depth-wise convolution layer to obtain feature maps processed by convolution; and perform convolution processing on all the feature maps processed by convolution through the point-wise convolution to obtain a feature map combined by channels.

9. An obstacle detection system characterized by, The image sensor is configured to acquire a detection image; the electronic control unit includes at least one processor and a memory; the memory stores computer execution instructions; ​ ​ ​ The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the obstacle detection method according to any one of claims 1 to 5.

10. A computer storage medium, characterized in that, The computer storage medium stores the computer-executable instructions, and when the processor executes the computer-executable instructions, the obstacle detection method according to any one of claims 1 to 5 is implemented.

Images