An object detection method, apparatus, device, and storage medium

Abstract

The application discloses an object detection method, device and equipment and a storage medium. The method comprises the following steps: performing target detection processing on a to-be-processed image to obtain an initial detection frame corresponding to a target object; performing region expansion on an initial region image based on preset proportion information to obtain an initial expansion region image containing the initial detection frame; performing region mapping processing on the initial expansion region image to obtain a mapping region image containing a mapping detection frame; obtaining target offset information of the target object compared with the mapping detection frame based on the mapping region image; and performing offset processing on the mapping detection frame based on the target offset information to determine the target detection frame containing the target object. The application can improve the accuracy of the target object detection frame, so that the target object detection frame is attached to the edge of the target object, and the target object detection accuracy is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image detection, in particular to an object detection method, device, equipment and storage medium. BACKGROUND

[0002] In order to further improve the accuracy of target detection, a higher definition camera (such as 8 million pixels) can be used. At present, the commonly used camera in the field of automatic driving captures images with a resolution of 1920*1080, 3840*2160, etc. When a target detection network based on deep learning is used for vehicle target detection, the image often needs to be scaled (such as commonly used 640*640) to reduce the calculation amount, so as to ensure the inference speed of the network. Due to the scaling, the target in the distance is small, and for a vehicle far away, the target detection box may have a certain deviation, which cannot well fit the edge of the vehicle, is easy to misjudge, and cannot accurately identify the vehicle type, license plate and other information at the same time, so a method for improving the accuracy of vehicle detection box is needed. SUMMARY

[0003] In order to overcome the shortcomings and deficiencies of the prior art, the present application discloses an object detection method, device, equipment and storage medium, which can improve the accuracy of the target object detection box, so that the target object detection box fits the edge of the target object, and further improves the target object detection accuracy. The method comprises:

[0004] performing target detection processing on the to-be-processed image to obtain an initial detection box corresponding to a target object;

[0005] performing region expansion on the initial region image based on preset proportion information to obtain an initial expansion region image containing the initial detection box; the initial region image is an image region corresponding to the initial detection box in the to-be-processed image;

[0006] performing region mapping processing on the initial expansion region image to obtain a mapping region image containing a mapping detection box; the size of the mapping region image is a preset image size, and the size of the mapping detection box is a preset detection box size; the proportion information of the preset image size and the preset detection box size meets the preset proportion information;

[0007] obtaining target offset information of the target object relative to the mapping detection box based on the mapping region image;

[0008] performing offset processing on the mapping detection box based on the target offset information to determine the target detection box containing the target object; the target detection box is more fitted to the edge of the target object than the mapping detection box.

[0009] Further, the method further comprises:

[0010] determine an offset interval based on the size information of the mapping bounding box and preset scale information; the preset scale information is determined based on a ratio of a size of the initial bounding box to a size of the initial region image;

[0011] The target offset information of the target bounding box object relative to the mapping bounding box is obtained based on the mapping region image, and the target offset information includes:

[0012] The target offset information is obtained based on the mapping region image and the offset interval of the mapping bounding box; and the target offset information is located in the offset interval.

[0013] Further, the target offset information is obtained based on the mapping region image and the offset interval of the mapping bounding box, and the target offset information includes:

[0014] An object detection network is obtained; the object detection network includes a feature extraction layer and an output layer;

[0015] The mapping region image is subjected to feature extraction based on the feature extraction layer to obtain image feature information;

[0016] The image feature information and preset scale information are input into the output layer, and the output layer is subjected to data processing to obtain the target offset information.

[0017] Further, the feature extraction layer includes multiple hierarchical feature extraction layers, wherein a low-level feature extraction layer is used to extract feature information corresponding to a first recognition type, and a high-level feature extraction layer is used to extract feature information corresponding to a second recognition type; a scale of the first recognition type in the to-be-processed image is greater than a scale of the second recognition type in the to-be-processed image.

[0018] The mapping region image is subjected to feature extraction based on the feature extraction layer to obtain image feature information, and the image feature information includes:

[0019] A target level is determined based on a type of the target object;

[0020] The mapping region image is subjected to feature extraction based on the feature extraction layer of the target level to obtain the image feature information.

[0021] Further, the target level includes a first level corresponding to the first recognition type and a second level corresponding to the second recognition type; and the image feature information includes first image feature information output by the first level and second image feature information output by the second level.

[0022] The feature extraction layer based on the target level extracts features of the mapping region image to obtain the image feature information.

[0023] The first image feature information is processed based on the output layer corresponding to the first level to obtain the target offset information.

[0024] The second image feature information is subjected to object recognition based on the output layer corresponding to the second level to obtain a sub-object contained in the target object.

[0025] In another aspect, the present application also provides an object detection device, comprising:

[0026] A processing module is configured to perform target detection processing on a to-be-processed image to obtain an initial detection frame corresponding to a target object.

[0027] A region expansion module is configured to perform region expansion on an initial region image based on preset proportion information to obtain an initial expansion region image containing the initial detection frame; the initial region image is an image region corresponding to the initial detection frame in the to-be-processed image.

[0028] A region mapping module is configured to perform region mapping processing on the initial expansion region image to obtain a mapping region image containing a mapping detection frame; the region mapping processing is performed on the initial expansion region image to obtain the mapping region image containing the mapping detection frame; the size of the mapping region image is a preset image size, and the size of the mapping detection frame is a preset detection frame size; the proportion information of the preset image size and the preset detection frame size satisfies the preset proportion information.

[0029] A target offset information generation module is configured to obtain target offset information of a target object relative to the mapping detection frame based on the mapping region image.

[0030] An offset processing module is configured to perform offset processing on the mapping detection frame based on the target offset information to determine the target detection frame containing the target object; the target detection frame is more fitted to the edge of the target object than the mapping detection frame.

[0031] In a third aspect, the present application also provides an electronic device, which comprises a processor and a memory; the memory stores at least one instruction, at least one program, a code set or an instruction set; the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement an object detection method as described above.

[0032] In a fourth aspect, the present application also provides a computer readable storage medium, wherein the storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to perform the object detection method as described above.

[0033] The present application has the following beneficial effects:

[0034] The present application performs object detection processing on the to-be-processed image to obtain an initial detection frame corresponding to the target object. In order to make the initial detection frame have a larger correction space, the initial region image is regionally expanded to obtain an initial expanded image. The initial detection frame containing the initial image is subjected to region mapping processing to obtain a mapping region image containing a mapping detection frame. After the region mapping processing, the size of the initial detection frame in different to-be-processed images is mapped to a preset detection frame size, and the size of the initial region image in different to-be-processed images is mapped to a preset image size. Based on the mapping detection frame with a unified size and the mapping region image with a unified size, the target offset information compared with the mapping detection frame is obtained based on the preset detection frame size. This can reduce the complexity of data processing. The mapping detection frame is subjected to offset processing based on the target offset information to obtain a target detection frame. Compared with the initial detection frame, the target detection frame is more fitted to the edge of the target object, thereby improving the detection accuracy of the target object. BRIEF DESCRIPTION OF DRAWINGS

[0035] In order to more clearly illustrate the technical solutions of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without any creative effort.

[0036] Figure 1 The object detection method flowchart provided for the embodiments of the present application;

[0037] Figure 2 The to-be-processed image provided for the embodiments of the present application;

[0038] Figure 3 The mapping region image provided for the embodiments of the present application;

[0039] Figure 4 The target detection frame schematic diagram provided for the embodiments of the present application;

[0040] Figure 5 The target offset information determination method flowchart provided for the embodiments of the present application;

[0041] Figure 6 The deep learning model feature extraction principle diagram provided for the embodiments of the present application;

[0042] Figure 7 A structure block diagram of an object detection device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0043] In order for those skilled in the art to better understand the technical solutions in the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should fall within the scope of protection of the present application.

[0044] In the present embodiment, the technical problem to be solved by the present application is to improve the accuracy of the target object detection frame, so as to make the target object detection frame fit the edge of the target object, and thus improve the target object detection accuracy.

[0045] Referring to Figure 1 The method comprises:

[0046] S110: performing target detection processing on the to-be-processed image to obtain an initial detection frame corresponding to the target object;

[0047] The execution subject of the present embodiment is a vehicle body processor or a server, which captures a to-be-processed image in real time through a vehicle-mounted camera. The to-be-processed image can also be an image stored in the cloud. The to-be-processed image includes various target detection objects, such as pedestrians, vehicles, traffic signs, etc., as shown in the following to-be-processed image: Figure 2 The to-be-processed image shown in the figure has a vehicle target object framed.

[0048] In the present embodiment, the target object is identified in the to-be-processed image through a target detection method. The target detection processing adopts a target detection algorithm of a YOLOv5 model. The YOLOv5 has a faster training speed, and the YOLOv5s has a smaller model size, which is beneficial to the rapid deployment of the model. As shown in the following to-be-processed image: Figure 2 The to-be-processed image shown in the figure is detected by the target detection algorithm of the YOLOv5 model to detect a vehicle in the to-be-processed image, and the vehicle target object is marked by an initial detection frame, so as to facilitate the extraction of feature information of the target object, such as the vehicle type, license plate, etc., from the initial region image corresponding to the initial detection frame.

[0049] It is known in the prior art that the initial detection frame has a certain precision, but when the target object is far away from the detection point, the initial detection frame may have a certain deviation, cannot well fit the edge of the target object, and even the target object may not be completely contained in the initial detection frame, which may cause incomplete feature extraction of the target object, such as too much initial detection frame exceeding the edge of the target object, too much irrelevant interference data of the feature data of the target object, increased difficulty of feature extraction of the target object, and reduced detection precision of the target object.

[0050] Therefore, the detection frame of the target object should fit the edge of the target object to ensure complete feature extraction of the target object and reduce the difficulty of feature extraction of the target object, so the initial detection frame needs to be adjusted to make the detection frame of the target object fit the edge of the target object.

[0051] S120: regionally expanding the initial region image based on preset proportion information to obtain an initial expansion region image containing the initial detection frame; the initial region image is an image region corresponding to the initial detection frame in the to-be-processed image;

[0052] In the to-be-processed image, the image in the initial detection frame is the initial region image. Generally, there are two cases of the initial detection frame, one is that the target object is completely in the initial detection frame, and the other is that the target object exceeds the initial detection frame. For the purpose of making the detection frame of the target object more fit the edge of the target object, the initial detection frame needs to be offset. The initial detection frame includes two offset cases, one is offset towards the initial region image, and the other is offset towards the opposite direction of the initial region image. If the initial region image is directly cut from the to-be-processed image, the initial detection frame can only offset towards the initial region image, and there is not enough space to offset the initial detection frame towards the opposite direction of the initial region image. Therefore, the initial region image needs to be regionally expanded based on a preset proportion to obtain an initial expansion region image. The initial detection frame is located in the initial expansion region image. In the initial expansion region image, the initial detection frame can offset towards the initial region image and offset towards the opposite direction of the initial region image to ensure that the detection frame of the target object fits the edge of the target object. If the initial expansion region image exceeds the range of the to-be-processed image, the exceeding region is filled with 0.

[0053] In the embodiment, the preset proportion information is 0.125 times or 0.25 times of the height and width of the initial detection frame, as an optimal implementation, the preset proportion information is 0.125, that is, the initial region image needs to be expanded by 0.125 times of the height and width of the initial detection frame in the up-down and left-right directions of the to-be-processed image based on the center point of the initial detection frame, that is, the upper edge of the initial region image is expanded upward by 0.125 times of the height of the initial detection frame, the lower edge of the initial region image is expanded downward by 0.125 times of the height of the initial detection frame, the left edge of the initial region image is expanded leftward by 0.125 times of the width of the initial detection frame, and the right edge of the initial region image is expanded rightward by 0.125 times of the width of the initial detection frame, and the expanded image region is an initial expansion region image. Therefore, the height and width of the initial expansion region image are 1.25 times of the height and width of the initial region image, where 1.25 = 1 + 0.125 * 2.

[0054] In order to extract the feature information of the target object from the to-be-processed image, the initial expansion region image is cropped from the to-be-processed image, and the initial expansion region image is obtained. Figure 3 This facilitates subsequent object recognition only on the initial expansion region image, thereby reducing the amount of calculation to ensure processing speed.

[0055] S130: performing region mapping processing on the initial expansion region image to obtain a mapping region image containing a mapping detection frame; the size of the mapping region image is a preset image size, and the size of the mapping detection frame is a preset detection frame size; the proportion information of the preset image size and the preset detection frame size meets the preset proportion information.

[0056] The mapping region image shown in Figure 3 The detection frame in Figure 3 is a mapping detection frame, and the outside of the mapping detection frame is an expanded region; through the region mapping processing, the size of the initial detection frame in different to-be-processed images is mapped to the preset detection frame size, and the size of the initial region image in different to-be-processed images is mapped to the preset image size, and the proportion information of the preset image size and the preset detection frame size meets the preset proportion information, that is, the distance between the mapping detection frame and the mapping region image and the proportion information of the preset detection frame size are equal to the preset proportion information, for example, the preset detection frame size is 128*128, the preset image size is 160*160, the distance between the mapping detection frame and the mapping region image is 16, and the ratio of 16 to 128 is equal to the preset proportion information, that is, 0.125, and the preset expansion parameter is 16 pixels. In the embodiment, only the target offset information of the target object relative to the mapping detection frame needs to be determined, the complexity of processing is reduced, and the inference speed of the target offset information is ensured.

[0057] S140: obtaining target offset information of the target object relative to the mapping detection frame based on the mapping region image.

[0058] The mapping region image is input into the deep learning model to perform feature extraction on the mapping region image to obtain target offset information compared with the mapping detection frame. To obtain the target offset information compared with the mapping detection frame, the deep learning model needs to be trained in advance. Training samples are input into the to-be-trained model, the size of the training samples is consistent with the size of the mapping region image, the target detection frame is manually calibrated, the target offset information is determined based on the target detection frame, the training samples are input into the to-be-trained model, the to-be-trained model generates offset information, loss information is determined according to the target offset information and the offset information, the parameters of the to-be-trained model are adjusted based on the loss information, a plurality of training samples are input into the to-be-trained model for iterative training until the loss information is less than a loss threshold, the training is completed, and the deep learning model is obtained. The deep learning model input into the mapping region image can output the target offset information.

[0059] S150: offset processing is performed on the mapping detection frame based on the target offset information to determine a target detection frame containing the target object; the target detection frame is more fitted to the edge of the target object compared with the mapping detection frame.

[0060] In the case where the mapping region image completely contains the target object, offset processing is performed on the mapping detection frame based on different target offset information to obtain a target detection frame corresponding to the target object. Since the target detection frame is closer to the edge of the target object compared with the mapping detection frame, the target detection frame is more fitted to the edge of the target object compared with the initial detection frame, and the detection accuracy of the target object is improved.

[0061] In an embodiment, the method further comprises:

[0062] The offset interval is determined based on the mapping detection frame and preset proportion information; the preset proportion information is determined based on the ratio information of the size of the initial detection frame to the size of the initial region image;

[0063] There can be multiple target detection objects in a mapping area image, for example, there are a large vehicle and a small vehicle in front of a detection point, the large vehicle is in front, and the small vehicle is behind, it is not determined whether the output target offset information corresponds to the large vehicle or the small vehicle, if the target offset information is not range-constrained, the output target offset information can be positive infinity or negative infinity, and the deep learning model is difficult to converge, therefore, before determining the target offset information, the offset interval needs to be determined, and the target offset information is determined based on the offset interval, the offset interval is determined according to the mapping detection frame and preset scale information, for example, the coordinates of the mapping detection frame in the mapping area image are [16, 16, 144, 144], the detection frame size is 128x128, the preset image size is 160x160, the preset scale information is 0.125, and the preset expansion parameter is 16 (128x0.125), therefore, the mapping detection frame can be maximally offset outward by 16, and maximally offset inward by 16, therefore, the offset interval is 0-32, the coordinates of the target detection frame in the mapping area image are [0, 0, 160, 160] when the size is maximum, the detection frame with the coordinates [0, 0, 160, 160] is determined as a first detection frame, and the coordinates of the detection frame with the minimum size are [32, 32, 128, 128], the detection frame with the coordinates [32, 32, 128, 128] is determined as a second detection frame.

[0064] Based on the mapping area image, target offset information of the target detection frame relative to the mapping detection frame is obtained, including:

[0065] Based on the mapping area image and the offset interval of the mapping detection frame, the target offset information is obtained; the target offset information is located in the offset interval.

[0066] Referring to Figure 4 , the detection frames from outside to inside are the first detection frame, the mapping detection frame, the target detection frame and the second detection frame in sequence, the distance between the first detection frame and the second detection frame is d, d is 2 times of the preset expansion pixel, that is, 32 pixel points are twice of 16 pixel points, the target offset information is determined in the offset interval, further, the target detection frame is located between the first detection frame and the second detection frame; before determining the target offset information, the offset interval is determined, which can accurately output the target offset information corresponding to the target object when multiple objects appear in the mapping area image, and improve the accuracy of the target detection frame.

[0067] In an implementation manner, based on the mapping area image, target offset information of the target detection frame relative to the mapping detection frame is obtained, referring to Figure 5 , including:

[0068] S510: An object detection network is acquired; the object detection network includes a feature extraction layer and an output layer;

[0069] The deep learning model comprises an object detection network, the object detection network comprises a Backbone network (main network) for extracting basic features, a FPN (Feature Pyramid Networks), and a FAN (Path Aggregation Network), and the Backbone network, the FPN, and the FAN each comprise a plurality of feature extraction layers.

[0070] S520: performing feature extraction on the mapping area image based on the feature extraction layers to obtain image feature information;

[0071] The mapping area image is input into the feature extraction layers, as shown in FIG. 2, the Backbone is a basic feature extraction network; the FPN is a kind of top-down feature pyramid, which can pass down the high-level strong semantic features and enhance the entire pyramid. The PAN adds a bottom-up pyramid based on the FPN, which can supplement the semantic information of the FPN and pass up the low-level positioning features, so that the object detection network can integrate semantic information and positioning information, and obtain the image feature information corresponding to the mapping area image. Figure 6

[0072] S530: inputting the image feature information and the preset scale information into an output layer, performing data processing based on the output layer to obtain target offset information.

[0073] Further, in the embodiment, the calculation formula of the target offset information is as follows:

[0074] y t,b,l,r =d*(sigmoid(x t,b,l,r )-0.5)

[0075] wherein x t,b,l,r is the output of the last feature extraction layer, i.e., the image feature information (feature vector output by the last feature extraction layer), and d is twice the preset expansion parameter; based on the preset expansion parameter, the offset range can be obtained, the image feature information and the preset expansion parameter are input into the output layer to obtain the target offset information. Figure 6 In the formula, Head is a detection head, which is an output layer, and the output content of the output layer includes four offset amounts corresponding to the mapping detection frame, i.e., the target offset information, and C (the type number of object types), which can be two types of non-vehicles and motor vehicles, or multiple categories such as non-vehicles, trucks, SUVs, and small cars. N in the figure is the number of pictures for one inference; the mapping area image is subjected to feature extraction by the multiple feature extraction layers, the accuracy and comprehensiveness of the image feature information are improved, and thus the accuracy of subsequent determination of the target detection frame is improved.

[0076] ​In an embodiment, the feature extraction layer includes multiple levels of feature extraction layers, specifically, the Backbone, the FPN and the FAN each include a low level of feature extraction layer and a high level of feature extraction layer, wherein the low level of feature extraction layer is used to extract feature information corresponding to a first recognition type, and the high level of feature extraction layer is used to extract feature information corresponding to a second recognition type; the scale of the first recognition type in the to-be-processed image is greater than the scale of the second recognition type in the to-be-processed image; correspondingly, the image feature information includes first image feature information and second image feature information, and the feature information output based on the high level of feature extraction layer is the first image feature information; the feature information output based on the low level of feature extraction layer is the second image feature information.

[0077] The feature extraction layer is used to extract features of the mapping region image to obtain image feature information, including:

[0078] Based on the type of the target object, a target level is determined.

[0079] Based on the feature extraction layer of the target level, the mapping region image is extracted to obtain image feature information.

[0080] Based on the image feature information, the type of the target object is determined, and the type of the target object includes a first recognition type and a second recognition type, the first recognition type includes recognition of an entire vehicle, and the second recognition type includes a license plate or a vehicle feature with a smaller size, if the target object is the first recognition type, a high level of feature extraction layer (i.e. Figure 6 The output of the PAN top layer) is determined as the target level, the high level of feature extraction layer has rich semantic information and is suitable for extracting a vehicle contour, etc., the first image feature information and a preset expansion parameter are input into the output layer to obtain target offset information corresponding to the first recognition type, without performing multi-scale output, and the complexity of the deep learning model is also reduced; if the target object is the second recognition type, a low level of feature extraction layer (i.e. Figure 6 The output of the PAN bottom layer) is determined as the target level, the low level of feature extraction layer extracts features such as contour, edge, color, texture and shape, but the range of the region that can be recognized is not as good as that of the high level of feature extraction layer, is suitable for detecting vehicle identification information with a smaller size, and can accurately extract a license plate number, the high level of feature extraction layer has rich semantic information and is suitable for extracting a vehicle contour, etc., can ensure complete recognition of the vehicle contour, and the accuracy of the target offset information is improved, the second image feature information and a preset expansion parameter are input into the output layer, and license plate information can be accurately obtained, different recognition types are used to determine the target level for feature extraction, the comprehensiveness and accuracy of the image feature information are ensured, and the complexity of the model is reduced.

[0081] In an implementation, the target level includes a first level corresponding to the first recognition type and a second level corresponding to the second recognition type; the image feature information includes first image feature information output by the first level and second image feature information output by the second level;

[0082] After the image feature information is obtained by performing feature extraction on the mapping area image based on the feature extraction layer of the target level, the method further includes:

[0083] Performing data processing on the first image feature information based on an output layer corresponding to the first level to obtain target offset information;

[0084] Performing object recognition on the second image feature information based on an output layer corresponding to the second level to obtain a sub-object contained in the target object.

[0085] The first level corresponds to a high-level feature extraction layer, which is suitable for images of a first recognition type. The second level corresponds to a low-level feature extraction layer, which is suitable for images of a second recognition type. A head is arranged at the first level and the second level respectively. The first image feature information is obtained by performing feature extraction on the mapping area image through the feature extraction layer of the first level. The second image feature information is obtained by performing feature extraction on the mapping area image through the feature extraction layer of the second level. When the type of the target object is the first recognition type, the first image feature information is processed by the head corresponding to the first level, and the target offset information is output. When the type of the target object is the second recognition type, the second image feature information is processed by the head corresponding to the second level. The output layers corresponding to different levels can be different, and the corresponding data processing functions can also be different. For example, some output layers can be used for offset calculation, and some output layers can be used for object recognition classification. The target object includes a sub-object. For example, the target object is a vehicle with license plate information. The vehicle contour is the first recognition type, and the license plate information is the second recognition type. The target offset information corresponding to the mapping detection frame is determined. The first recognition type performs feature extraction on the mapping area image through the feature extraction layer of the first level. The head performs data processing based on the first image feature information output by the first level, to obtain the target offset information corresponding to the mapping detection frame, and further obtain the target detection frame of the vehicle through the target offset information. In addition, the license plate information is a sub-object of the vehicle, and the license plate information is the second recognition type. The second recognition type performs feature extraction on the mapping area image through the feature extraction layer of the second level. The feature extraction layer of the second level has high recognition accuracy and is suitable for extracting license plate information. The head performs license plate information recognition based on the second image feature information output by the second level based on the second image feature information, to obtain the license plate information of the vehicle. The embodiment can perform multi-scale head output on the same target object, and perform classification recognition on the target object and the sub-object of the target object. Different recognition types perform feature extraction through feature extraction layers of different levels, which improves the data processing efficiency and reduces the complexity of the model.

[0086] In an embodiment, the mapping detection frame is offset processed based on the target offset information to determine the target detection frame containing the target object, including:

[0087] The coordinate information of the mapping detection frame in the mapping area image is added to the target offset information to obtain the coordinate information of the target detection frame in the mapping area image.

[0088] Reference Figure 6The target offset information is four offset vectors of the coordinate information of the mapping detection frame in the mapping region image. The coordinate information of the mapping detection frame in the mapping region image is added to the target offset information to obtain the coordinate information of the target detection frame in the mapping region image. As can be seen from the figure, the target detection frame is closer to the edge of the vehicle than the mapping detection frame, and the accuracy of the target detection frame is improved.

[0089] The embodiment also provides an object detection device which can implement all the method steps described above, as shown in the figure, the device comprises: Figure 7

[0090] The processing module 710 is configured to perform target detection processing on the to-be-processed image based on preset proportion information to obtain an initial detection frame corresponding to the target object.

[0091] The region expansion module 720 is configured to perform region expansion on the initial region image to obtain an initial expansion region image containing the initial detection frame.

[0092] The region mapping module 730 is configured to perform region mapping processing on the initial expansion region image to obtain a mapping region image containing a mapping detection frame.

[0093] The target offset information generation module 740 is configured to obtain target offset information of the target object relative to the mapping detection frame based on the mapping region image.

[0094] The offset processing module 750 is configured to perform offset processing on the mapping detection frame based on the target offset information to determine a target detection frame containing the target object.

[0095] Further, the device further comprises:

[0096] The first determination module is configured to determine an offset interval based on the mapping detection frame and preset expansion parameters.

[0097] The second processing module is configured to obtain the target offset information based on the mapping region image and the offset interval of the mapping detection frame.

[0098] ​The acquisition module is configured to acquire an object detection network; the object detection network comprises a feature extraction layer and an output layer;

[0099] The feature extraction module is configured to perform feature extraction on the mapping region image based on the feature extraction layer to obtain image feature information.

[0100] The input module is configured to input the image feature information and preset scale information into the output layer, perform data processing based on the output layer, and obtain target offset information.

[0101] The second determination module is configured to determine a target level feature extraction layer based on a type of the target object; the feature extraction layer comprises a plurality of level feature extraction layers, wherein a low level feature extraction layer is used to extract feature information corresponding to a first recognition type, and a high level feature extraction layer is used to extract feature information corresponding to a second recognition type; a scale of the first recognition type in the to-be-processed image is greater than a scale of the second recognition type in the to-be-processed image.

[0102] The third determination module is configured to determine the feature information of the target level feature extraction layer as the image feature information.

[0103] The third processing module is configured to perform data processing on the first image feature information based on the output layer corresponding to the first level to obtain target offset information.

[0104] The fourth processing module is configured to perform object recognition on the second image feature information based on the output layer corresponding to the second level to obtain a sub-object contained in the target object.

[0105] The fifth processing module is configured to obtain coordinate information of the target detection frame in the mapping region image based on a sum of coordinate information of the mapping detection frame in the mapping region image and the target offset information.

[0106] The implementation of the embodiment has the following effects:

[0107] 1. The present application performs object detection processing on the to-be-processed image to obtain an initial detection frame corresponding to the target object. In order to make the initial detection frame have a larger correction space, the initial region image is expanded to obtain an initial expanded image. The initial detection frame containing the initial image is subjected to region mapping processing to obtain a mapping region image containing a mapping detection frame. After the region mapping processing, the size of the initial detection frame in different to-be-processed images is fixed to a uniform size, and the size of the initial region image in different to-be-processed images is fixed to a uniform size. Based on the uniform size of the mapping detection frame and the uniform size of the mapping region image, target offset information compared with the mapping detection frame is obtained. The mapping detection frame is subjected to offset processing based on the target offset information to obtain a target detection frame. The target detection frame is more fitted to the edge of the target object compared with the initial detection frame, thereby improving the detection accuracy of the target object.

[0108] 2. The size of different initial detection boxes is enlarged / reduced by a preset multiple, and mapped to a uniform size, that is, the detection box is mapped, and the size of different initial region images is enlarged / reduced by a preset multiple, and mapped to a uniform size, that is, the region image is mapped, only the target offset information compared with the mapped detection box needs to be determined, the target detection box closest to the edge of the target object can be determined, and the complexity of processing can be reduced.

[0109] Embodiments of the present application also provide an electronic device, which comprises a processor and a memory, and the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to realize an object detection method as in the method embodiments.

[0110] Embodiments of the present application also provide a storage medium, which can be arranged in a server to save at least one instruction, at least one program, a code set or an instruction set for realizing an object detection method as in the method embodiments, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to realize an object detection method provided in the above method embodiments.

[0111] Optionally, in the embodiment, the storage medium can be located in at least one of a plurality of network servers of a computer network. Optionally, in the embodiment, the storage medium can include but is not limited to a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk and various storage program codes.

[0112] It should be noted that the terms "first", "second" and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or server including a series of steps or units does not necessarily limit to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to the process, method, product or device.

[0113] The foregoing description has been set forth in terms of specific embodiments of the application. It is to be understood that modifications which do not depart from the scope of the application as set forth in the claims will occur to those skilled in the art upon a reading of the foregoing description. Accordingly, the scope of the application is to be interpreted only as is delineated in the claims.

Claims

1. A method of object detection, characterized by, The method comprises: performing object detection processing on a to-be-processed image to obtain an initial detection box corresponding to a target object; performing region expansion on an initial region image based on preset proportion information to obtain an initial expanded region image containing the initial detection box; the initial region image is an image region corresponding to the initial detection box in the to-be-processed image; performing region mapping processing on the initial expanded region image to obtain a mapping region image containing a mapping detection box; the size of the mapping region image is a preset image size, and the size of the mapping detection box is a preset detection box size; the proportion information of the preset image size and the preset detection box size satisfies the preset proportion information; based on the mapping region image, target offset information of the target object relative to the mapping detection box is obtained; based on the target offset information, offset processing is performed on the mapping detection box to determine a target detection box containing the target object; the target detection box is more fitted to the edge of the target object than the mapping detection box.

2. The object detection method of claim 1, wherein, The method further comprises: determining an offset interval based on the size information of the mapping detection box and the preset proportion information; the preset proportion information is determined based on the ratio information of the size of the initial detection box to the size of the initial region image; based on the mapping region image and the offset interval of the mapping detection box, the target offset information is obtained; the target offset information is located in the offset interval. based on the mapping region image and the offset interval of the mapping detection box, the target offset information is obtained; the target offset information is located in the offset interval.

3. The object detection method of claim 2, wherein, obtain an object detection network; the object detection network comprises a feature extraction layer and an output layer; based on the feature extraction layer, feature extraction is performed on the mapping region image to obtain image feature information; the image feature information and the preset proportion information are input into the output layer, and data processing is performed based on the output layer to obtain the target offset information. The feature extraction layer comprises a plurality of hierarchical feature extraction layers, wherein a low-level feature extraction layer is used to extract feature information corresponding to a first recognition type, and a high-level feature extraction layer is used to extract feature information corresponding to a second recognition type; the image scale corresponding to the first recognition type is greater than the image scale corresponding to the second recognition type; 4. The object detection method of claim 3, wherein, based on the feature extraction layer, feature extraction is performed on the mapping region image to obtain image feature information, comprising: determining a target level based on the type of the target object; based on the feature extraction layer of the target level, feature extraction is performed on the mapping region image to obtain the image feature information. The target level comprises a first level corresponding to the first recognition type and a second level corresponding to the second recognition type; the image feature information comprises first image feature information output by the first level and second image feature information output by the second level; 5. The object detection method of claim 4, wherein, after the feature extraction layer of the target level is used to perform feature extraction on the mapping region image to obtain the image feature information, the method further comprises: ​ perform data processing on the first image feature information based on an output layer corresponding to the first level, to obtain the target offset information; perform object recognition on the second image feature information based on an output layer corresponding to the second level, to obtain a sub-object contained in the target object.

6. The object detection method of claim 1, wherein The offset processing on the mapping detection frame based on the target offset information to determine the target detection frame containing the target object includes: obtaining coordinate information of the target detection frame in the mapping region image based on a sum of coordinate information of the mapping detection frame in the mapping region image and the target offset information.

7. An object detection device, characterized by, includes: The processing module is configured to perform target detection processing on a to-be-processed image to obtain an initial detection frame corresponding to a target object. The region expansion module is configured to perform region expansion on an initial region image based on preset proportion information to obtain an initial expanded region image containing the initial detection frame; the initial region image is an image region corresponding to the initial detection frame in the to-be-processed image. The region mapping module is configured to perform region mapping processing on the initial expanded region image to obtain a mapping region image containing a mapping detection frame; the region mapping processing on the initial expanded region image is performed to obtain the mapping region image containing the mapping detection frame; the mapping region image has a preset image size, and the mapping detection frame has a preset detection frame size. The proportion information of the preset image size and the preset detection frame size satisfies the preset proportion information. The target offset information generation module is configured to obtain target offset information of a target object relative to the mapping detection frame based on the mapping region image. The offset processing module is configured to perform offset processing on the mapping detection frame based on the target offset information to determine a target detection frame containing the target object. The target detection frame is more fitted to an edge of the target object than the mapping detection frame.

8. An electronic device, comprising: The device includes a processor and a memory, and the memory stores at least one instruction, at least one program, a code set, or an instruction set, which are loaded and executed by the processor to implement the object detection method of any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The storage medium stores at least one instruction, at least one program, a code set, or an instruction set, which are loaded and executed by the processor to implement the object detection method of any one of claims 1 to 6.

Images