An object detection method and system based on an occupation network and a storage medium

Abstract

The application relates to an object detection method and system based on an occupation network and a storage medium, the method comprising L1. A vehicle travels on a road, real-time image data information around the vehicle is acquired based on a vehicle-mounted all-around camera, image splicing and scaling processing are carried out, and processed image data information is output; L2. The processed image data information is input into an occupation network object detection model for detection, and occupation data information of a 3D space of an image is output; and L3. The occupation data information of the 3D space of the image is subjected to smoothing processing, discrete points are eliminated, and processed occupation data information of the 3D space of the image is output. The application not only realizes detection of objects with unclear shapes in a scene, but also can predict three-dimensional information, the information is more abundant, and reliable information support can be provided for downstream tasks.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of target recognition, in particular to a target detection method and system based on an occupancy network and a storage medium. BACKGROUND

[0002] In the past few years, autonomous driving technology has been widely developed, especially in the aspect of vehicle perception, and significant breakthroughs have been made. At present, in the field of autonomous driving, traditional target detection methods use Multi-Camera to collect videos around the vehicle body, perceive the distance, size, speed, density, etc. of objects around the vehicle body, and then perform motion planning. However, the traditional target detection method cannot perceive the contour information of the surrounding objects, and is easily affected by the environment, resulting in false detection or missed detection, so it may fail in open-world traffic scenarios where the shape or appearance of the target is not clear. These objects with unclear shape or appearance are called long-tail obstacles, including deformable obstacles such as two-section trailers, irregular obstacles such as overturned vehicles, unknown category obstacles such as road debris and garbage, and partially occluded objects, etc. SUMMARY

[0003] In view of the above problems, the present application provides a target detection method and system based on an occupancy network and a storage medium, which not only realizes the detection of objects with unclear shape in the scene, but also predicts three-dimensional information, which is more informative and can provide reliable information support for downstream tasks.

[0004] To achieve the above objects and other related objects, the technical solutions provided by the present application are as follows:

[0005] A target detection method based on an occupancy network, the method comprising:

[0006] L1. A vehicle is driving on a road, based on a vehicle-mounted all-around camera, real-time image data information around the vehicle is obtained, and the image is spliced and scaled to output processed image data information;

[0007] L2. The processed image data information is input into an occupancy network target detection model for detection, and occupancy data information of the image 3D space is output;

[0008] L3. The occupancy data information of the image 3D space is smoothed to eliminate discrete points, and the processed occupancy data information of the image 3D space is output;

[0009] L4. Based on the processed image 3D space occupancy data information, a boundary calculation method is used to calculate the distance of the target in the image, and distance data information of the target is output;

[0010] L5. generating Bev information of the target based on the distance data information of the target, and outputting target perception data information according to the Bev information of the target.

[0011] Further, in step L2, the inputting the processed image data information into the occupancy network target detection model for detection comprises:

[0012] L21. inputting the processed image data information into a Resnet34 convolution network for image feature extraction, fusing high-level features of a deep convolution layer and low-level features of a shallow convolution layer, and outputting image feature tensor data information;

[0013] L22. inputting the image feature tensor data information into a 2DTo3D network for image conversion, and outputting 3D data information of the image;

[0014] L23. inputting the 3D data information of the image into a 3DVGG network for classification, and outputting occupancy data information of a 3D space of the processed image.

[0015] Further, the high-level features of the deep convolution layer are deep convolution layer features of the image, and the low-level features of the shallow convolution layer are shallow convolution layer features of the image.

[0016] Further, the occupancy data information of the 3D space of the processed image comprises a plurality of targets, and each target is composed of a plurality of 3D rectangular units.

[0017] Further, in step L4, the distance calculation of the target in the image by using the boundary calculation method comprises:

[0018] L41. calculating a 3D rectangular unit with the largest distance from the vehicle body as max xyz and a 3D rectangular unit with the smallest distance from the vehicle body as min xyz based on the occupancy data information of the 3D space of the processed image.

[0019] L42. obtaining a maximum circumscribed rectangle Rect hwh and a center coordinate Center xyz of the target according to the max xyz and the min xyz .

[0020] Center xyz = 0.5 * (max xyz + min xyz ),

[0021] Rect hwh = max xyz - min xyz .

[0022] L43. According to the maximum circumscribed rectangle Rect of the target hwh and the center coordinates Center xyz , output the distance data information of the target.

[0023] Further, in step L5, the target perception data information includes relative position data information and target shape data information of the target.

[0024] To achieve the above object and other related objects, the present application further provides a target detection system based on an occupancy network, which comprises:

[0025] An image acquisition module, configured to acquire image data information around a vehicle in real time;

[0026] An image preprocessing module, connected with the image acquisition module, configured to perform scaling and splicing processing on the image;

[0027] An occupancy network target detection module, connected with the image preprocessing module, configured to perform classification and processing on the preprocessed image to obtain 3D occupancy information of the target;

[0028] A calculation module, connected with the occupancy network target detection module, configured to calculate distance data information of the target;

[0029] An output module, connected with the calculation module, configured to output perception data information of the target.

[0030] Further, the occupancy network target detection module comprises an Encoder unit and a Decoder unit, the Encoder unit is a Resnet34 convolution network, a deep convolution layer and a shallow convolution layer, and the Decoder unit comprises a 2DTo3D subunit and a 3DVGG subunit.

[0031] Further, the system further comprises a display module, which is connected with the output module and configured to display the perception data information of the target received in real time.

[0032] To achieve the above object and other related objects, the present application further provides a computer readable storage medium, which stores a computer program programmed or configured to execute any one of the target detection methods based on the occupancy network.

[0033] The present application has the following positive effects:

[0034] 1. The present application can not only display the surrounding obstacles in real time, but also has a relatively low cost, which is conducive to popularization and use.

[0035] 2. The application can predict three-dimensional information through the processing of images by occupying the network, the information is more abundant, and reliable information support can be provided for downstream tasks. BRIEF DESCRIPTION OF DRAWINGS

[0036] Figure 1 It is a flowchart of the method of the application;

[0037] Figure 2 It is a schematic diagram of the image stitching result of the application;

[0038] Figure 3 It is a schematic diagram of the Bev display effect of the application. DETAILED DESCRIPTION

[0039] The exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present disclosure to help understanding, and should be considered only as exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Also, for the sake of clarity and conciseness, the description below omits the description of well-known functions and structures.

[0040] Embodiment 1: as shown in Figure 1 or Figure 2 or Figure 3 A target detection method based on an occupancy network, the method comprising:

[0041] L1. The vehicle travels on the road, based on the vehicle-mounted all-around camera, real-time acquisition of image data information around the vehicle, and image stitching and scaling processing, output of the processed image data information;

[0042] L2. The processed image data information is input into an occupancy network target detection model for detection, and the occupancy data information of the image 3D space is output;

[0043] L3. The occupancy data information of the image 3D space is smoothed to eliminate discrete points, and the processed occupancy data information of the image 3D space is output;

[0044] L4. Based on the processed image 3D space occupancy data information, a boundary calculation method is used to calculate the distance of the target in the image, and the distance data information of the target is output;

[0045] L5. Based on the distance data information of the target, the Bev information of the target is generated, and the target perception data information is output according to the Bev information of the target.

[0046] In the embodiment, in step L2, the inputting the processed image data information into the network target detection model for detection comprises:

[0047] L21. inputting the processed image data information into a Resnet34 convolution network for image feature extraction, fusing high-level features of deep convolution layers and low-level features of shallow convolution layers, and outputting image feature tensor data information;

[0048] L22. inputting the image feature tensor data information into a 2DTo3D network for image conversion, and outputting 3D data information of the image;

[0049] L23. inputting the 3D data information of the image into a 3DVGG network for classification, and outputting occupancy data information of the processed image 3D space.

[0050] In the embodiment, the high-level features of the deep convolution layers are deep convolution layer features of the image, and the low-level features of the shallow convolution layers are shallow convolution layer features of the image.

[0051] In the embodiment, the occupancy data information of the processed image 3D space comprises a plurality of targets, and each target is composed of a plurality of 3D cuboid units.

[0052] In the embodiment, in step L4, the distance calculation of the target in the image by using the boundary calculation method comprises:

[0053] L41. based on the occupancy data information of the processed image 3D space, calculating a 3D cuboid unit farthest from the vehicle body as max xyz and a 3D cuboid unit closest to the vehicle body as min xyz ;

[0054] L42. according to the max xyz and min xyz , obtaining a maximum circumscribed rectangle Rect hwh and a center coordinate Center xyz of the target,

[0055] Center xyz = 0.5*(max xyz + min xyz ),

[0056] Rect hwh = max xyz -min xyz ;

[0057] L43. according to the maximum circumscribed rectangle Rect hwh and the center coordinate Centerxyz outputting distance data information of the target.

[0058] In this embodiment, in step L5, the target awareness data information includes relative position data information and target shape data information of the target.

[0059] Embodiment 1: Based on a target detection method of an occupancy network in embodiment 1, the present application is further described and illustrated below.

[0060] The present application provides a target detection system based on an occupancy network, which comprises:

[0061] An image acquisition module is configured to acquire image data information around a vehicle in real time.

[0062] An image preprocessing module is connected to the image acquisition module and configured to perform scaling and splicing processing on the image.

[0063] An occupancy network target detection module is connected to the image preprocessing module and configured to classify and process the preprocessed image to obtain 3D occupancy information of the target.

[0064] A calculation module is connected to the occupancy network target detection module and configured to calculate distance data information of the target.

[0065] An output module is connected to the calculation module and configured to output awareness data information of the target.

[0066] In this embodiment, the occupancy network target detection module comprises an Encoder unit and a Decoder unit, the Encoder unit is a Resnet34 convolution network, a deep convolution layer and a shallow convolution layer, and the Decoder unit comprises a 2DTo3D subunit and a 3DVGG subunit.

[0067] The Encoder unit mainly comprises a bottom-up path and a top-down path. The bottom-up path uses the Resnet34 convolution network to extract features, the top-down path performs up-sampling on the features of the last Resnet34 convolution network, and the lateral connection is a process of fusing features of the deep convolution layer and features of the shallow convolution layer. The high-level features of the deep convolution layer and the low-level features of the shallow convolution layer are outputted through the fusion.

[0068] In this embodiment, the system further comprises a display module connected to the output module and configured to display the awareness data information of the target received in real time.

[0069] The application provides a computer readable storage medium, which stores a computer program programmed or configured to perform any one of the target detection methods based on an occupation network.

[0070] Any reference to memory, storage, database, or other medium in the embodiments provided by the present application can include non-volatile and / or volatile memory. The non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. The volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct RAMbus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0071] In summary, the application not only realizes the detection of objects with unclear shapes in a scene, but also predicts three-dimensional information, which is more abundant and can provide reliable information support for downstream tasks.

[0072] The above detailed description does not constitute a limitation on the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modification, equivalent replacement, and improvement within the spirit and principle of the present disclosure should be included in the protection scope of the present disclosure.

Claims

1. A method for object detection based on an occupancy network, characterized in that, The method includes: L1. When a vehicle is driving on the road, it acquires real-time image data information of the vehicle's surroundings based on the vehicle's on-board cameras, and performs image stitching and scaling processing on the images, outputting the processed image data information. L2. Input the processed image data information into the occupancy network target detection model for detection, and output the occupancy data information of the image 3D space; L3. Smooth the 3D space occupancy data of the image to eliminate discrete points and output the processed 3D space occupancy data of the image; L4. Based on the processed image 3D space occupancy data, the boundary calculation method is used to calculate the distance to the target in the image and output the target distance data. L5. Based on the distance data of the target, generate the target's Bev information, and output the target perception data information according to the target's Bev information; In step L2, inputting the processed image data information into the occupies the network target detection model for detection includes: L21. Input the processed image data information into the ResNet34 convolutional network for image feature extraction, and fuse the high-level features of the deep convolutional layers and the low-level features of the shallow convolutional layers to output the image's feature tensor data information; L22. Input the tensor data of the image into a 2DTo3D network for image transformation, and output the 3D data of the image; L23. Input the 3D data information of the image into a 3DVGG network for classification, and output the 3D space occupancy data information of the processed image; In step L4, the distance calculation of the target in the image using the boundary calculation method includes: L41. Based on the processed image 3D space occupation data information, calculate the 3D cuboid unit farthest from the vehicle body as max xyz and the smallest 3D cuboid unit as min xyz ; L42. According to the max xyz and min xyz , get the maximum bounding rectangle Rect hwh and the center coordinates Center xyz of the target, Center xyz = 0.5 * (max xyz + min xyz ), Rect hwh =max xyz -min xyz ; L43. According to the maximum circumscribed rectangle Rect of the target hwh and the center coordinates Center xyz , output the distance data information of the target.

2. The occupancy network-based object detection method of claim 1, wherein: The high-level features of the deep convolutional layer are the deep convolutional layer features of the image, and the low-level features of the shallow convolutional layer are the shallow convolutional layer features of the image.

3. The occupancy network-based object detection method of claim 1, wherein: The occupancy data information of the processed image 3D space includes multiple targets, each of which is composed of multiple 3D rectangular units.

4. The occupancy network-based object detection method of claim 1, wherein, In step L5, the target perception data information includes the target's relative position data information and the target's shape data information.

5. An object detection system based on an occupancy network, characterized in that The system for implementing the target detection method based on occupancy networks according to any one of claims 1-4, the system comprising: The image acquisition module is used to acquire image data information about the vehicle's surroundings in real time. An image preprocessing module, connected to the image acquisition module, is used to scale and stitch images. An occupancy network target detection module, connected to the image preprocessing module, is used to classify and process the preprocessed image to obtain 3D occupancy information of the target. The processed image data information is then input into the occupancy network target detection model for detection, including: L21. Input the processed image data information into the ResNet34 convolutional network for image feature extraction, and fuse the high-level features of the deep convolutional layers and the low-level features of the shallow convolutional layers to output the image's feature tensor data information; L22. Input the tensor data of the image into a 2DTo3D network for image transformation, and output the 3D data of the image; L23. inputting 3D data information of the image into a 3DVGG network for classification, and outputting occupancy data information of a processed image 3D space; The computing module is connected with the occupancy network target detection module, and is configured to calculate distance data information of the target, and calculate the distance of the target in the image by using a boundary calculation method. L41. Based on the processed image 3D space occupation data information, calculate the 3D cuboid unit farthest from the vehicle body as max xyz and the smallest 3D cuboid unit as min xyz ; L42. According to the max xyz and min xyz , get the maximum bounding rectangle Rect of the target hwh and the center coordinates Center xyz , Center xyz = 0.5 * (max xyz + min xyz ), Rect hwh =max xyz -min xyz ; L43. According to the maximum circumscribed rectangle Rect of the target hwh and the center coordinates Center xyz , output the distance data information of the target; The output module is connected with the computing module, and is configured to output the perception data information of the target.

6. The occupancy network-based object detection system of claim 5, wherein: The occupancy network target detection module comprises an Encoder unit and a Decoder unit, the Encoder unit is a Resnet34 convolution network, a deep convolution layer and a shallow convolution layer, and the Decoder unit comprises a 2DTo3D subunit and a 3DVGG subunit.

7. The occupancy network-based object detection system of claim 5, wherein: The system further comprises a display module connected with the output module, and configured to display the perception data information of the target received in real time.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program programmed or configured to perform the occupancy network based target detection method in any one of claims 1-4.

Images