An object detection method and device, electronic equipment and storage medium

Abstract

The application provides an object detection method and device, electronic equipment and storage medium. An image including an object is obtained. The image is input into an object detection model, and a first object region image result is obtained from the image. The object detection model is trained based on an object training sample and a corresponding target true value. The object detection model includes a first feature extraction network and a second feature extraction network. The first feature extraction network realizes extraction of object region features through a neural network. The second feature extraction network takes the output of the first feature extraction network as input, realizes extraction of identification features of the object region by fusing features in the object region, and directly detects the object image by inputting the object image into the object detection model, thereby realizing end-to-end training and improving the object detection effect.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of deep learning, and in particular to an object detection method and device, electronic equipment and a storage medium. BACKGROUND

[0002] In the existing object detection method, there is a kind of method which is divided into two stages: object initial positioning is performed according to a traditional image processing method, and then an object detection model is trained by a neural network method. This kind of method is not an end-to-end method, and the result of the second stage depends on the first stage. The results of the two stages cannot be complementary to each other. If the initial positioning image is inaccurate when performing initial positioning, the object detection result is also inaccurate even if the second stage is to train an object detection model by a neural network method. SUMMARY

[0003] The present application provides an object detection method and device, electronic equipment and a storage medium, which directly detects by inputting an object image into an object detection model, realizes end-to-end training, improves the object detection effect, solves the problem of inaccurate object detection result in the existing object detection technology, and obtains an image including an object. The image is input into an object detection model, and a first object region image result is obtained from the image. The object detection model is trained based on an object training sample and a corresponding target true value. The first feature extraction network realizes extraction of object region features by a neural network. The second feature extraction network extracts identification features of the object region by taking the output of the first feature extraction network as input and fusing features in the object region.

[0004] In a first aspect, the present application provides an object detection method, which comprises:

[0005] An image including an object is obtained.

[0006] The image is input into an object detection model, and a first object region image result is obtained from the image. The object detection model is trained based on an object training sample and a corresponding target true value.

[0007] The object detection model comprises a first feature extraction network and a second feature extraction network. The first feature extraction network realizes extraction of object region features by a neural network. The second feature extraction network extracts identification features of the object region by taking the output of the first feature extraction network as input and fusing features in the object region.

[0008] Further, the image is input into the object detection model, and the first object region image result is obtained from the image, specifically comprising:

[0009] inputting the image into a first feature extraction network, and performing first feature extraction on the image based on the first feature extraction network, wherein the first feature extraction comprises: performing down-sampling on the image to obtain a first object feature; and performing up-sampling on the down-sampled first object feature to obtain a second object feature and a second object region image result corresponding to the second object feature;

[0010] performing feature fusion on the image and the second object feature based on a second feature extraction network, and performing second feature extraction, wherein the second feature extraction comprises: performing down-sampling on the feature after the feature fusion to obtain a third object feature;

[0011] performing feature fusion on the first object feature and the third object feature, and performing up-sampling to obtain a first object region image result.

[0012] Further, the object detection model is trained, comprising:

[0013] obtaining an object training sample and a corresponding target true value thereof;

[0014] using the object training sample as input data for training, using the target true value corresponding to the object training sample as a label, training the object model to be trained, and obtaining an object detection model for generating a first object region image result.

[0015] Further, the training of the object detection model to be trained comprises:

[0016] inputting the object training sample into the object detection model to be trained to obtain a first object region image result and a second object region image result output by the object detection model to be trained;

[0017] constructing a first object loss function according to the first object region image result and a target true value corresponding to the first object region image result;

[0018] constructing a second object loss function according to the second object region image result and a target true value corresponding to the second object region image result;

[0019] obtaining a total loss function based on the first object loss function and the second object loss function, and ending the training based on convergence of the total loss function.

[0020] Further, the object comprises any one of the following:

[0021] a license plate, a traffic sign, a vehicle, an obstacle, and a lane line.

[0022] In a second aspect, the present application further provides an object detection device, comprising an acquisition module and a detection module, wherein:

[0023] The acquisition module is configured to acquire an image comprising an object.

[0024] The detection module is configured to input the image into an object detection model and obtain a first object region image result from the image, wherein the object detection model is trained based on object training samples and corresponding target true values.

[0025] The object detection model comprises a first feature extraction network and a second feature extraction network; the first feature extraction network is configured to extract object region features through a neural network; the second feature extraction network is configured to extract identification features of the object region by taking the output of the first feature extraction network as input and fusing features in the object region.

[0026] In a third aspect, the present application further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the object detection method according to any one of the above aspects when executing the program.

[0027] In a fourth aspect, the present application further provides a non-transitory computer readable storage medium, having a computer program stored thereon, wherein the computer program is executable on a processor to implement the steps of the object detection method according to any one of the above aspects.

[0028] In a fifth aspect, the present application further provides a computer program product, comprising a computer program, wherein the computer program is executable on a processor to implement the steps of the object detection method according to any one of the above aspects.

[0029] The object detection method, device, electronic device, and storage medium provided by the present application acquire an image comprising an object; input the image into an object detection model and obtain a first object region image result from the image, wherein the object detection model is trained based on object training samples and corresponding target true values; the object detection model comprises a first feature extraction network and a second feature extraction network; the first feature extraction network extracts object region features through a neural network; the second feature extraction network takes the output of the first feature extraction network as input, fuses features in the object region, and extracts identification features of the object region only, directly detects the object image by inputting the object image into the object detection model, realizes end-to-end training, and improves the object detection effect. BRIEF DESCRIPTION OF DRAWINGS

[0030] In order to make the technical solutions in the present application or the prior art clearer, the accompanying drawings needed in the embodiments or prior art description will be briefly introduced. Obviously, the accompanying drawings in the following description are only some embodiments of the present application, and all other drawings obtained by those of ordinary skill in the art without creative work based on these drawings belong to the protection scope of the present application.

[0031] Figure 1 is a flowchart of an object detection method provided by an embodiment of the present application;

[0032] Figure 2 is a flowchart of an object detection method provided by another embodiment of the present application;

[0033] Figure 3 is a flowchart of an object detection method provided by another embodiment of the present application;

[0034] Figure 4 is a structural block diagram of an object detection device provided by an embodiment of the present application;

[0035] Figure 5 is a structural block diagram of an object detection electronic device provided by another embodiment of the present application. DETAILED DESCRIPTION

[0036] In order to make the technical solutions in the present application or the prior art clearer, the accompanying drawings needed in the embodiments or prior art description will be briefly introduced. Obviously, the accompanying drawings in the following description are only some embodiments of the present application, and all other drawings obtained by those of ordinary skill in the art without creative work based on these drawings belong to the protection scope of the present application.

[0037] Figure 1 is a flowchart of an object detection method provided by an embodiment of the present application, referring to Figure 1 The method comprises the following steps.

[0038] Step 101: acquiring an image comprising an object;

[0039] The execution subject of the object detection method in the embodiment of the present application can be an electronic device, further a processor of the electronic device. The electronic device can be a camera or other devices with image processing capability such as a computer device, etc., which is not limited here.

[0040] The image including the object is an image including a complete object captured by the camera in the monitoring scene, and the camera capturing the image including the object can include, for example, the following: during monitoring, the camera performs object detection on a monitoring picture, and when detecting that the monitoring picture includes an object, the camera captures an image to obtain the image including the object. Of course, the manner of obtaining the image including the object is not limited to this.

[0041] It should be further explained that the object includes any one of the following: a license plate, a traffic sign, a vehicle, an obstacle, and a lane line, and the object to be detected is not specifically limited herein.

[0042] Step 102: inputting the image into an object detection model and obtaining a first object region image result from the image;

[0043] The object detection model is trained based on object training samples and corresponding target true values.

[0044] The object detection model includes a first feature extraction network and a second feature extraction network; the first feature extraction network is implemented by a neural network to extract object region features; the second feature extraction network takes the output of the first feature extraction network as input, and extracts identification features of the object region by fusing features in the object region.

[0045] The object detection model is trained in advance and stored in a local or external device, and the object detection model can be called when step 102 is performed. The object detection model is implemented by a neural network.

[0046] It can be understood that the first feature extraction network is a BaseNet module, the second feature extraction network is a RefineNet module, and the first object region image result is obtained by training the image through the BaseNet module and the RefineNet module.

[0047] It should be further explained that the BaseNet module can be implemented by a convolutional neural network, but is not limited to this, to preliminarily extract object features; the RefineNet module takes the output of the BaseNet module as input, fuses features with the image, and further extracts object features, that is, the RefineNet module can better focus and learn the object region during training.

[0048] The embodiment provides an object detection method, acquires an image including an object; inputs the image into an object detection model, and obtains a first object region image result from the image; wherein the object detection model is obtained by training based on an object training sample and a corresponding target true value; the object detection model includes a first feature extraction network and a second feature extraction network; the first feature extraction network realizes extraction of object region features through a neural network; the second feature extraction network realizes extraction of identification features of the object region only by fusing features in the object region by taking the output of the first feature extraction network as input, directly extracts the identification features of the object region by inputting the object image into the object detection model for detection, and combines the two backbone networks together to realize end-to-end training, improve the object detection effect, and make the final object detection result more accurate.

[0049] Based on the content of the above embodiment, in the embodiment, the step 102 of inputting the image into the object detection model and obtaining a first object region image result from the image can be realized by the following manner:

[0050] Step 201: input the image into the first feature extraction network, and perform first feature extraction on the image based on the first feature extraction network;

[0051] The first feature extraction process includes: performing down-sampling on the image to obtain first object features; and performing up-sampling on the down-sampled first object features to obtain second object features and a second object region image result corresponding to the second object features.

[0052] It should be further explained that the main purpose of down-sampling is to reduce the image, and the step length of convolution is added between each convolution layer to reduce the size of the feature map output by each convolution layer, so as to reduce the calculation amount; the main purpose of up-sampling is to enlarge the image, so that the image after feature extraction is the same size as the original image, so as to ensure the normal performance of subsequent feature fusion, and the method used is an interpolation method, that is, on the basis of the original image pixels, a new element is inserted between the pixel point values by using a suitable interpolation algorithm.

[0053] In addition, the number of times of down-sampling and up-sampling is related to the technical effect to be achieved by the person skilled in the art, and is not specifically limited.

[0054] For example, the original image pixel size is 640*480, by setting the step length Stride=2, when the convolution kernel is 3*3 convolution, the output feature image size will become 1 / 2 of the original, that is, 320*240, then, on the basis of the feature image size of 320*240, further down-sampling is performed, and the size of the original image pixel is 1 / 4, that is, 160*120, on the basis of the feature image size of 160*120, further down-sampling is performed once, and the output feature image size will become 1 / 8 of the original image, that is, 80*60, the feature image with the size of 80*60 after feature extraction is up-sampled by 8 times, and the second object feature and the second object region image result are obtained.

[0055] Step 202: performing feature fusion on the image and the second object feature based on the second feature extraction network module and performing second feature extraction;

[0056] In the process of the second feature extraction, the feature after the feature fusion of the image and the second object feature is down-sampled to obtain a third object feature.

[0057] It can be understood that the feature fusion here is to extract the object feature information from the original image, that is, the second object feature mentioned above, and to analyze, process and integrate the second object feature with the original image to obtain the fused image feature, that is, the third object feature, and the accuracy of object detection on the fused feature is obviously higher than that of the original image.

[0058] It needs to be further explained that the feature after the feature fusion of the image and the second object feature is down-sampled, which is also to set the step length of convolution between the convolution layers to reduce the size of the feature map output by each convolution layer.

[0059] Specifically, the original image and the second object feature output by the BaseNet module are fused based on the RefineNet module and second feature extraction is performed.

[0060] For example, the image and the second object feature are fused based on the RefineNet module to obtain a fused image, and at this time, the pixel size of the fused image is 640*480, and the step length Stride=2 is set for down-sampling, when the convolution kernel is 3*3 convolution, the output feature image size will become 1 / 2 of the original, that is, 320*240, and the above steps are repeated, and down-sampling is performed twice to obtain a third object feature with the size of 1 / 8 of the original image.

[0061] Step 203: feature fusion is performed on the first object feature and the third object feature, and upsampling is performed to obtain a first object region image result.

[0062] The first object feature obtained by downsampling the image through the BaseNet module is again feature fused with the third object feature obtained by feature fusion and second feature extraction on the original image and the output of the BaseNet module through the RefineNet module, and then upsampling is performed to obtain a final first object region image result.

[0063] For example, the feature map obtained by downsampling the image with an original size of 640*480 through the BaseNet module has a size of 80*60, that is, the first object feature. The second object feature obtained by feature fusion and downsampling of the image with an original size of 640*480 and the output of the BaseNet module, which has a size of 640*480, has a size of 80*60. The first object feature and the third object feature, both of which have a pixel size of 80*60, are again feature fused and 8 times upsampling is performed to obtain a first object region image result with the same size as the original image, that is, the object image result finally detected.

[0064] The embodiment provides an object detection method. The image is input into a first feature extraction network. First feature extraction is performed on the original image based on the first feature extraction network. The process of the first feature extraction includes: downsampling the image to obtain a first object feature; upsampling the first object feature after the downsampling to obtain a second object feature and a second object region image result corresponding to the second object feature; and feature fusion is performed on the image and the second object feature based on a second feature extraction network, and second feature extraction is performed. The process of the second feature extraction includes: downsampling the feature after the feature fusion of the image and the second object feature to obtain a third object feature; and feature fusion is performed on the first object feature and the third object feature, and upsampling is performed to obtain a first object region image result. By combining the first feature extraction network and the second feature extraction network into two backbone networks, end-to-end training is realized, and information complementation is realized through feature fusion and input fusion, so that the object detection effect is improved, and the final object detection result is more accurate.

[0065] Based on the content of the above embodiment, in this embodiment, the object detection model is trained, including:

[0066] An object training sample and a corresponding target true value are obtained.

[0067] The object training sample is taken as input data for training, and the target true value corresponding to the object training sample is taken as a label, and the object detection model to be trained is trained to obtain an object detection model for generating a first object region image result.

[0068] In this embodiment, the object training sample and the corresponding target true value are obtained, including: obtaining an image containing an object, and using an object detection method to screen out an image containing object information as an effective training sample; and labeling the effective training sample to obtain a label and corresponding bounding box information of the object. When labeling the effective training sample, that is, when the object appears in the image, the bounding box is marked, and the label is set as the object accordingly.

[0069] This embodiment provides an object detection method, which obtains an object training sample and a corresponding target true value; takes the object training sample as input data for training, takes the target true value corresponding to the object training sample as a label, trains an object detection model to be trained, and obtains an object detection model for generating a first object region image result, that is, the establishment of the object detection model is realized.

[0070] Based on the content of the above embodiment, in this embodiment, the training of the object detection model to be trained includes:

[0071] The object training sample is input into the object detection model to be trained to obtain a first object region image result and a second object region image result output by the object detection model to be trained;

[0072] A first object loss function is constructed according to the first object region image result and the target true value corresponding to the first object region image result;

[0073] A second object loss function is constructed according to the second object region image result and the target true value corresponding to the second object region image result;

[0074] A total loss function is obtained based on the first object loss function and the second object loss function, and the training is ended based on the convergence of the total loss function.

[0075] It should be further explained that the object detection model to be trained is trained according to the SGD algorithm.

[0076] In this embodiment, the training is ended based on the convergence of the total loss function until the total loss function is less than or equal to a first preset value. The total loss function can be used to evaluate the optimization degree of the object detection model. The smaller the total loss function value is, the more accurate the object detection model is. The first preset value is set by a person skilled in the art according to the actual application requirements of the model, and is not specifically limited here.

[0077] The embodiment provides an object detection method. Object training samples are input into a to-be-trained object model to obtain first object region image results and second object region image results output by the to-be-trained object model; a first object loss function is constructed according to the first object region image results and target true values corresponding to the first object region image results; a second object loss function is constructed according to the second object region image results and target true values corresponding to the second object region image results; a total loss function is obtained based on the first object loss function and the second object loss function, and training is ended based on convergence of the total loss function. By continuously converging the total loss function, the object detection model is optimized, and the first object region image results obtained finally are more accurate, that is, the object detection effect is improved.

[0078] Figure 3 is a flowchart of an object detection method provided by another embodiment of the application, referring to Figure 3 In the embodiment, the object is a specific position license plate. A collected vehicle image is acquired, the vehicle image is input into a BaseNet module, the vehicle image is subjected to first downsampling based on the BaseNet module to obtain a feature map with a size of 1 / 2 of an original vehicle image; the feature map with the size of 1 / 2 of the original vehicle image is subjected to second downsampling to obtain a feature map with a size of 1 / 4 of the original vehicle image; the feature map with the size of 1 / 4 of the original vehicle image is subjected to third downsampling to obtain a feature map with a size of 1 / 8 of the original vehicle image, that is, a first license plate feature; the feature map with the size of 1 / 8 of the original vehicle image is subjected to 8 times upsampling to obtain a second license plate feature with the same size as the original vehicle image and after preliminary feature extraction; the second license plate feature, that is, the output of the BaseNet module, is subjected to feature fusion with the original vehicle image as input of a RefineNet module; the feature map after feature fusion is subjected to first downsampling to obtain a feature map with a size of 1 / 2 of the original vehicle image; the feature map with the size of 1 / 2 of the original vehicle image is subjected to second downsampling to obtain a feature map with a size of 1 / 4 of the original vehicle image; the feature map with the size of 1 / 4 of the original vehicle image is subjected to third downsampling to obtain a feature map with a size of 1 / 8 of the original vehicle image, that is, a third license plate feature; finally, the first license plate feature and the third license plate feature are subjected to feature fusion again and 8 times upsampling to obtain a final license plate detection result.

[0079] Figure 4 A structural block diagram of an object detection device provided by the embodiment is shown in the figure, the device comprises an acquisition module 401 and a detection module 402, wherein:

[0080] The acquisition module 401 is configured to acquire an image comprising an object.

[0081] The execution subject of the object detection method in the embodiment of the present application can be an electronic device, further a processor of the electronic device, and the electronic device can be a camera or other devices with image processing capability such as a computer device, which is not limited here.

[0082] The image including the object is an image including a complete object collected by the camera in a monitoring scene, and the camera collecting the image including the object can include the following: the camera performs object detection on a monitoring picture in a monitoring process, and captures the image to obtain the image including the object when detecting that the monitoring picture includes the object. Of course, the manner of obtaining the image including the object is not limited to this.

[0083] It should be further explained that the object includes any one of the following: a license plate, a traffic sign, a vehicle, an obstacle and a lane line, and the detection object is not limited here.

[0084] The detection module 402 is configured to input the image into an object detection model and obtain a first object region image result from the image.

[0085] The object detection model is trained based on object training samples and corresponding target true values.

[0086] The object detection model includes a first feature extraction network and a second feature extraction network. The first feature extraction network is implemented by a neural network to extract object region features. The second feature extraction network takes the output of the first feature extraction network as input, and extracts identification features of the object region by fusing features in the object region.

[0087] The object detection model is trained in advance and stored in a local or external device. The object detection model is called by the detection module 402 during detection, and the object detection model is implemented by a neural network.

[0088] It can be understood that the first feature extraction network is a BaseNet module, the second feature extraction network is a RefineNet module, and the first object region image result is obtained by training the image through the BaseNet module and the RefineNet module.

[0089] It should be further explained that the BaseNet module can be implemented by a convolutional neural network, but is not limited to this, to preliminarily extract object features; the RefineNet module takes the output of the BaseNet module as input, fuses features with the image, and further extracts object features, that is, the RefineNet module can better focus and learn the object region during training.

[0090] The embodiment provides an object detection device, an acquisition module 401 is used for acquiring an image including an object; a detection module 402 is used for inputting the image into an object detection model and obtaining a first object region image result from the image; wherein the object detection model is obtained by training based on an object training sample and a corresponding target true value; the object detection model comprises a first feature extraction network and a second feature extraction network; the first feature extraction network is used for extracting object region features through a neural network; the second feature extraction network is used for extracting identification features of the object region only by fusing features in the object region through taking the output of the first feature extraction network as input; the object image is directly input into the object detection model for detection; two backbone networks are combined together to realize end-to-end training, improve the object detection effect, and make the final object detection result more accurate.

[0091] The detection module 402 specifically comprises:

[0092] The image is input into the first feature extraction network, and the first feature extraction network is used for performing first feature extraction on the image, wherein the process of the first feature extraction comprises: performing down sampling on the image to obtain first object features; and performing up sampling on the down-sampled first object features to obtain second object features and a second object region image result corresponding to the second object features;

[0093] The second feature extraction network is used for performing feature fusion on the image and the second object features and performing second feature extraction, wherein the process of the second feature extraction comprises: performing down sampling on the features after the feature fusion of the image and the second object features to obtain third object features;

[0094] The first object features and the third object features are fused and up-sampled to obtain the first object region image result.

[0095] The embodiment provides an object detection device, which is used for inputting an image into a first feature extraction network, performing first feature extraction on the image based on the first feature extraction network, wherein the process of the first feature extraction comprises: performing down sampling on the image to obtain first object features; performing up sampling on the down-sampled first object features to obtain second object features and a second object region image result corresponding to the second object features; performing feature fusion on the image and the second object features based on a second feature extraction network and performing second feature extraction, wherein the process of the second feature extraction comprises: performing down sampling on the features after the feature fusion on the image and the second object features to obtain third object features; performing feature fusion on the first object features and the third object features and performing up sampling to obtain a first object region image result. By combining the first feature extraction network and the second feature extraction network together, end-to-end training is realized, and information complementation is realized through feature fusion and input fusion, the object detection effect is improved, and the final object detection result is more accurate.

[0096] Optionally, the device further comprises:

[0097] The object training sample and the corresponding target true value are obtained.

[0098] The object training sample is used as input data for training, the target true value corresponding to the object training sample is used as a label, a to-be-trained object detection model is trained, and an object detection model for generating a first object region image result is obtained.

[0099] The embodiment provides an object detection device, which obtains an object training sample and a corresponding target true value, uses the object training sample as input data for training, uses the target true value corresponding to the object training sample as a label, trains a to-be-trained object detection model, and obtains an object detection model for generating a first object region image result, so that the establishment of the object detection model is realized.

[0100] Optionally, the device further comprises:

[0101] The object training sample is input into the to-be-trained object detection model to obtain a first object region image result and a second object region image result output by the to-be-trained object detection model.

[0102] A first object loss function is constructed according to the first object region image result and a target true value corresponding to the first object region image result.

[0103] A second object loss function is constructed according to the second object region image result and a target true value corresponding to the second object region image result.

[0104] Based on the first object loss function and the second object loss function, the total loss function is obtained, and the training ends when the total loss function converges.

[0105] This embodiment provides an object detection device for inputting object training samples into a model to be trained, obtaining a first object region image result and a second object region image result output by the model; constructing a first object loss function based on the first object region image result and the target ground truth value corresponding to the first object region image result; constructing a second object loss function based on the second object region image result and the target ground truth value corresponding to the second object region image result; obtaining a total loss function based on the first object loss function and the second object loss function, and ending training based on the convergence of the total loss function. By continuously converging the total loss function, the object detection model is optimized, making the final first object region image result more accurate, thus improving the object detection effect.

[0106] Figure 5 This is a schematic diagram of the physical structure of an electronic device provided in an embodiment of the present invention, such as... Figure 5 As shown, the electronic device may include a processor 510, a communication interface 520, a memory 530, and a bus 540. The processor 510, communication interface 520, and memory 530 communicate with each other via the bus 540. The bus 540 can be used for information transmission between the electronic device and sensors. The processor 510 can call logical instructions in the memory 530 to execute the following methods: acquiring an image including an object; inputting the image into an object detection model and obtaining a first object region image result from the image; wherein the object detection model is trained based on object training samples and their corresponding target ground values; the object detection model includes a first feature extraction network and a second feature extraction network; the first feature extraction network extracts object region features through a neural network; the second feature extraction network, by taking the output of the first feature extraction network as input, extracts only the identifying features of the object region through feature fusion within the object region.

[0107] In addition, the logic instructions in the memory 530 described above can be implemented in the form of software functional units and sold or used as independent products, and can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the parts that contribute to the prior art or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0108] In another aspect, the embodiments of the present application provide a non-transitory computer readable storage medium storing computer instructions, which cause a computer to execute the object detection method provided by the above-mentioned embodiments, for example, comprising: obtaining an image including an object; inputting the image into an object detection model and obtaining a first object region image result from the image; wherein the object detection model is trained based on object training samples and their corresponding target true values; the object detection model comprises a first feature extraction network and a second feature extraction network; the first feature extraction network realizes extraction of object region features through a neural network; the second feature extraction network takes the output of the first feature extraction network as input, and extracts identification features of the object region by fusing features in the object region.

[0109] From the above description of the embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software plus necessary universal hardware platforms, and of course can also be realized by hardware. Based on such understanding, the above technical solutions essentially or the parts that contribute to the prior art can be embodied in the form of a software product. The computer software product can be stored in a computer readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments.

[0110] In yet another aspect, the present application also provides a computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions that, when executed by a computer, enable the computer to perform an object detection method as described above, the method comprising: obtaining an image comprising an object; inputting the image into an object detection model and obtaining a first object region image result from the image; wherein the object detection model is trained based on object training samples and corresponding target true values; the object detection model comprises a first feature extraction network and a second feature extraction network; the first feature extraction network extracts object region features through a convolutional network; the second feature extraction network takes the output of the first feature extraction network as input, and extracts identification features of the object region by fusing features in the object region.

[0111] Finally, it should be noted that the above examples are only used to illustrate the technical solutions of the present application, and are not intended to limit the same; although the present application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that they can still modify the technical solutions described in the foregoing examples, or make equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method of object detection, characterized by, The method comprises the following steps: acquiring an image comprising an object; inputting the image into an object detection model and obtaining a first object region image result from the image; wherein the object detection model is trained based on an object training sample and a corresponding target true value; the object detection model comprises a first feature extraction network and a second feature extraction network; the first feature extraction network realizes extraction of object region features through a neural network; the second feature extraction network realizes extraction of identification features of the object region by taking the output of the first feature extraction network as input and fusing features in the object region; training the object detection model comprises the following steps: acquiring an object training sample and a corresponding target true value; taking the object training sample as input data for training and taking the target true value corresponding to the object training sample as a label to train the object detection model to be trained to obtain an object detection model for generating a first object region image result, specifically comprising the following steps: inputting the object training sample into the object detection model to be trained to obtain a first object region image result and a second object region image result output by the object detection model to be trained; the second object region image result is an object recognition result output by the first feature extraction network after the image is input into the first feature extraction network; constructing a first loss function according to the first object region image result and the target true value corresponding to the first object region image result; constructing a second loss function according to the second object region image result and the target true value corresponding to the second object region image result; obtaining a total loss function based on the first loss function and the second loss function, and ending the training based on convergence of the total loss function.

2. The object detection method according to claim 1, characterized in that, The method comprises the following steps: inputting the image into the first feature extraction network to perform first feature extraction on the image based on the first feature extraction network, wherein the process of first feature extraction comprises: performing down-sampling on the image to obtain first object features; performing up-sampling on the down-sampled first object features to obtain second object features and a second object region image result corresponding to the second object features; performing feature fusion on the image and the second object features based on the second feature extraction network and performing second feature extraction, wherein the process of second feature extraction comprises: performing down-sampling on the features after the feature fusion on the image and the second object features to obtain third object features; performing feature fusion on the first object features and the third object features and performing up-sampling to obtain the first object region image result.

3. The method of claim 1 or 2, wherein the object comprises any one of the following: a license plate, a traffic sign, a vehicle, an obstacle, and a lane line.

4. An object detection device, characterized by, The method comprises the following steps: acquiring an image comprising an object; The detection module is configured to input the image into an object detection model and obtain a first object region image result from the image; the object detection model is trained based on object training samples and corresponding target true values; The object detection model comprises a first feature extraction network and a second feature extraction network; the first feature extraction network is configured to extract object region features by using a neural network; the second feature extraction network is configured to extract identification features of the object region by fusing features in the object region and taking the output of the first feature extraction network as input; The object detection model is trained by: obtaining object training samples and corresponding target true values; taking the object training samples as input data for training, taking the target true values corresponding to the object training samples as labels, training the object detection model to be trained, and obtaining an object detection model for generating a first object region image result, specifically including: inputting the object training samples into the object detection model to be trained to obtain a first object region image result and a second object region image result output by the object detection model to be trained; the second object region image result is an object recognition result output by the first feature extraction network after the image is input into the first feature extraction network; constructing a first loss function according to the first object region image result and the target true value corresponding to the first object region image result; constructing a second loss function according to the second object region image result and the target true value corresponding to the second object region image result; based on the first loss function and the second loss function, obtaining a total loss function, and ending the training based on the convergence of the total loss function.

5. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to realize the steps of the object detection method according to any one of claims 1 to 3. 6.A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to realize the steps of the object detection method according to any one of claims 1 to 3.

7. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to realize the steps of the object detection method according to any one of claims 1 to 3.

Images