Target detection method and apparatus

Abstract

The application provides a target detection method and device, the method comprising: obtaining a to-be-detected image and inputting the to-be-detected image into a pre-trained target detection model to obtain a target detection result of the to-be-detected image output by the detection model; the target detection model performs target detection on the to-be-detected image based on a preset first category object and a preset second category object, and obtains the target detection result of the to-be-detected image by merging the detection results of the first category object and the second category object. The application classifies and detects the to-be-recognized object in the to-be-detected image, performs target detection on the to-be-detected image based on the preset first category object and the preset second category object, and can effectively improve the accuracy and efficiency of target detection.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a target detection method and apparatus. Background Technology

[0002] Object detection, also known as object extraction, is an image segmentation technique based on the geometric and statistical features of objects. With the development of computer technology and the widespread application of computer vision principles, research on real-time target tracking using computer image processing techniques has become increasingly popular. Dynamic real-time target tracking and localization has broad application value in intelligent transportation systems, intelligent monitoring systems, military target detection, and surgical instrument positioning in medical navigation surgeries. For example, in certain specific monitoring scenarios, such as autonomous driving scenarios, it is necessary to detect the type of target object and its bounding box in road images acquired in order to issue timely warnings.

[0003] The target detection algorithms currently used include one-stage target detection algorithms and two-stage target detection algorithms. One-stage detection algorithms have fast detection speed but low detection accuracy, while two-stage detection algorithms have high detection accuracy but slow detection speed. When performing target detection, only one of these algorithms can be used to identify and detect objects in the image. It is difficult to balance detection speed and detection accuracy between the two, and it is impossible to simultaneously meet the requirements of fast detection speed and high accuracy. Summary of the Invention

[0004] To address the problems existing in the prior art, the present invention provides an adaptive holographic functional screen modulation method and apparatus.

[0005] This invention provides a target detection method, comprising:

[0006] The image to be detected is acquired and input into a pre-trained target detection model to obtain the target detection result output by the detection model of the image to be detected;

[0007] The target detection model performs target detection on the image to be detected based on a preset first category of objects and a preset second category of objects, and merges the detection results of the first category of objects and the second category of objects to obtain the target detection result of the image to be detected.

[0008] According to a target detection method provided by the present invention, the target detection model includes a first detection module and a second detection module. The first detection module is used to acquire target detection results of a preset first category of objects in the image to be detected, and the second detection module is used to acquire target detection results of a preset second category of objects in the image to be detected.

[0009] Wherein, the detection accuracy of the first category of objects in the preset classification and detection model is higher than the preset accuracy threshold, and the detection accuracy of the second category of objects in the preset classification and detection model is lower than the preset accuracy threshold.

[0010] According to a target detection method provided by the present invention, the first detection module includes:

[0011] The first feature extraction layer is used to extract the primary features of the image to be detected;

[0012] The first prediction layer is used to obtain the target detection result of the first category of objects based on the primary features.

[0013] According to a target detection method provided by the present invention, the first feature extraction layer includes:

[0014] The backbone network is used to extract multi-level features from the image to be detected;

[0015] The feature fusion network is used to obtain primary features of the image to be detected based on the multi-level features extracted by the backbone network.

[0016] According to a target detection method provided by the present invention, the second detection module includes:

[0017] The second feature extraction layer is used to extract aggregated features from the image to be detected;

[0018] The second prediction layer is used to obtain the target detection results of the second category of objects based on the aggregated features.

[0019] According to a target detection method provided by the present invention, the second detection module includes:

[0020] The second feature extraction layer is used to extract aggregate features of the input image to be detected based on the image to be detected and the primary features output by the first detection module.

[0021] The second prediction layer is used to obtain the target detection results of the second category of objects based on the aggregated features.

[0022] According to a target detection method provided by the present invention, the second feature extraction layer includes:

[0023] A region candidate network is used to obtain the target box prediction results of the second category of objects based on the primary features extracted by the first detection module;

[0024] The feature aggregation network is used to obtain aggregated features based on the bounding box prediction results and the image to be detected.

[0025] According to a target detection method provided by the present invention, the second feature extraction layer includes:

[0026] A region candidate network is used to obtain the target box prediction results of the second category of objects based on the primary features extracted by the first detection module;

[0027] The original image extraction network is used to obtain the features of the original image based on the target bounding box prediction results and the image to be detected.

[0028] The feature aggregation network is used to obtain aggregated features based on the target bounding box prediction results and the features of the original image.

[0029] The present invention also provides a target detection device, comprising:

[0030] The target detection unit is used to acquire the image to be detected and input it into a pre-trained target detection model to obtain the target detection result of the image to be detected output by the detection model;

[0031] The target detection model performs target detection on the image to be detected based on a preset first category of objects and a preset second category of objects, and merges the detection results of the first category of objects and the second category of objects to obtain the target detection result of the image to be detected.

[0032] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement any of the target detection methods described above.

[0033] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the target detection method as described above.

[0034] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the target detection method as described above.

[0035] The target detection method and apparatus provided by the present invention pre-classify the category of the object to be detected, extract primary features by the first detection module and detect the type and bounding box of the first category of objects in the image to be detected, and extract aggregate features by the second detection module and detect the type and bounding box of the second category of objects in the image to be detected. The detection of the corresponding object by the first detection module and the second detection module can effectively improve the speed and accuracy of target detection. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0037] Figure 1 This is a flowchart illustrating the target detection method provided by the present invention;

[0038] Figure 2 This is one of the schematic diagrams of the target detection method architecture provided by the present invention;

[0039] Figure 3 This is the second schematic diagram of the target detection method architecture provided by the present invention;

[0040] Figure 4 This is the third schematic diagram of the target detection method architecture provided by the present invention;

[0041] Figure 5 This is a schematic diagram of the target detection device provided by the present invention;

[0042] Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0044] Figure 1 A flowchart of a target detection method according to the present invention is shown, the method comprising:

[0045] The image to be detected is acquired and input into a pre-trained target detection model to obtain the target detection result output by the detection model of the image to be detected;

[0046] The object detection model performs object detection on the image to be detected based on a preset first category of objects and a preset second category of objects, and then merges the detection results of the first category of objects and the second category of objects to obtain the object detection result of the image to be detected.

[0047] The following is combined Figure 2-5 The target detection method of the present invention is described.

[0048] In one alternative embodiment, such as Figure 2 As shown, the target detection model includes a first detection module 201 and a second detection module 202. The first detection module 201 is used to obtain the target detection results of a preset first category of objects in the image to be detected, and the second detection module 202 is used to obtain the target detection results of a preset second category of objects in the image to be detected.

[0049] Among them, the detection accuracy of the first category of objects in the preset classification and detection model is higher than the preset accuracy threshold, while the detection accuracy of the second category of objects in the preset classification and detection model is lower than the preset accuracy threshold.

[0050] This invention takes an autonomous driving scenario as an example, detecting the type and bounding box of target objects in acquired road images to facilitate timely warnings. The target detection method of this invention is not limited to the aforementioned autonomous driving scenario; it can also be applied to other scenarios requiring object category and bounding box detection in images, such as target detection in surveillance video images and infrared sensor images.

[0051] In autonomous driving scenarios, the target objects to be detected include buses, trucks, trailers, cars, construction vehicles, traffic cones, motorcycles, bicycles, and pedestrians. Buses, trucks, trailers, and cars are pre-classified as Category 1 objects, while construction vehicles, traffic cones, motorcycles, bicycles, and pedestrians are pre-classified as Category 2 objects. The classification of Category 1 and Category 2 objects is based on the difficulty of recognition.

[0052] Specifically, in this invention, pre-trained one-stage object detection algorithms such as YOLOv1, YOLOv3, SSD, and RetinaNet, or pre-trained two-stage object detection algorithms such as R-CNN and Fast R-CNN are used as classification and detection models to perform object detection on all objects to be identified. Objects with detection accuracy higher than the accuracy threshold are selected as the first category of objects, and objects with detection accuracy lower than the accuracy threshold are selected as the second category of objects.

[0053] The first detection module 201 and the second detection module 202 of the present invention can detect objects of the first category and objects of the second category, respectively. The first detection module 201 detects objects with low detection difficulty, and the second detection module 202 detects objects with high detection difficulty. The speed and accuracy of target detection can be improved by using classification detection.

[0054] In one alternative embodiment, such as Figure 3 As shown, the first detection module 201 includes: a first feature extraction layer 301, used to extract primary features of the image to be detected; and a first prediction layer 302, used to obtain the target detection result of a first category of objects based on the primary features.

[0055] In this invention, the first detection module 201 directly extracts the primary features of the image to be detected through the first feature extraction layer 301 and uses the first prediction layer 302 to obtain the target detection results of the first category of objects. Based on this detection method, the target detection speed of the first category of objects can be effectively improved while ensuring the detection accuracy of the first category of objects, thereby improving the overall detection speed of the target detection model.

[0056] Furthermore, in an optional embodiment, the first feature extraction layer 301 includes: a backbone network for extracting multi-level features of the image to be detected; and a feature fusion network for obtaining primary features of the image to be detected based on the multi-level features extracted by the backbone network.

[0057] Specifically, the backbone network can adopt backbone networks such as VGG network and ResNet, and the feature fusion network can adopt a neck network structure such as FPN and PANet. The above backbone networks are only examples. The specific forms of the backbone network and feature fusion network of the present invention are not limited to the above network forms. The first prediction layer 302 is the prediction head. The first prediction layer 302 outputs the target detection results corresponding to the first category of objects, including the predicted target center point corresponding to the first category of objects, the width and height of the predicted target box corresponding to the first category of objects, and the object category corresponding to each predicted target box corresponding to the first category of objects.

[0058] In one alternative embodiment, such as Figure 2 As shown, the second detection module 202 consists of two detection modules set in parallel with the first detection module, which do not interfere with each other. The second detection module 202 includes:

[0059] The second feature extraction layer is used to extract aggregated features from the image to be detected;

[0060] The second prediction layer is used to obtain the target detection results of the second category of objects based on the aggregated features.

[0061] Specifically, the second detection module 202 of the present invention can adopt the network architecture of existing two-stage target detection networks, such as the R-CNN series neural network structure, for target detection. The second detection module 202 can acquire the aggregated features of the image to be detected and perform target detection. The second detection module 202 can detect objects that are difficult to detect and can improve the speed and accuracy of target detection by using classification detection.

[0062] In one alternative embodiment, such as Figure 4 As shown, the second detection module 202 is a continuation of the first detection module 201, and the second detection module 202 includes:

[0063] The second feature extraction layer is used to extract aggregate features of the input image to be detected based on the image to be detected and the primary features output by the first detection module 201.

[0064] The second prediction layer is used to obtain the target detection results of the second category of objects based on the aggregated features.

[0065] The second feature extraction layer of the second detection module 202 of the present invention performs aggregated feature extraction based on the image to be detected and the primary features output by the first detection module 201. It can perform feature extraction based on the calculation of the first detection module 201, further improve the efficiency of obtaining aggregated features, improve the speed of target detection, and use aggregated features to achieve high-precision detection of the second category of objects.

[0066] Furthermore, in an alternative embodiment, such as Figure 3 As shown, the second feature extraction layer includes:

[0067] The region candidate network 303 is used to obtain the target box prediction results of the second category of objects based on the primary features extracted by the first detection module 201.

[0068] Feature aggregation network 304 is used to obtain aggregated features based on the bounding box prediction results and the image to be detected.

[0069] Among them, the region candidate network 303 is an RPN network, which can extract the target box prediction result based on the primary feature. The feature aggregation network 304 is an ROI Align network, which combines the feature data of the image to be detected and the target box prediction result to obtain aggregated features. Then, the obtained aggregated features are processed by the encoder and sent to the second prediction layer. The second prediction layer is the prediction head, which obtains the target detection result of the second category of objects based on the aggregated features.

[0070] The second feature extraction layer of the present invention obtains the target box prediction result based on the region candidate network 303 and performs feature aggregation based on the feature aggregation network, which can improve the reliability of feature acquisition and achieve high-precision detection of the second category of objects.

[0071] In one alternative embodiment, such as Figure 4 As shown, the second feature extraction layer includes:

[0072] The region candidate network 303 is used to obtain the target box prediction results of the second category of objects based on the primary features extracted by the first detection module 201.

[0073] The original image extraction network 401 is used to obtain the features of the original image based on the target bounding box prediction results and the image to be detected.

[0074] Feature aggregation network 304 is used to obtain aggregated features based on the target bounding box prediction results and the features of the original image.

[0075] The region candidate network 303 is an RPN network, capable of extracting bounding box predictions based on primary features. The original image extraction network 401 is a backbone network. The bounding box predictions and the image to be detected are fed into the original image extraction network 401 to obtain original image features. Then, the original image features and the bounding box predictions are fed together into the feature aggregation network 304 for processing to obtain aggregated features. The obtained aggregated features are then processed by the encoder and fed into the second prediction layer, which is the prediction head. Based on the aggregated features, the target detection results for the second category of objects are obtained.

[0076] This invention utilizes an original image extraction network to further extract the original image features of the image to be detected, which can improve the detection accuracy of the second type of object.

[0077] In an optional embodiment, the object detection method of the present invention is a pre-trained object detection model. Its training steps are the same as those of existing neural network models. The parameters of the neural network model are trained and adjusted by constructing a training image set and a validation set until the model's loss function converges. Taking the loss function including the loss function of the region candidate network 303, the loss of the first prediction layer 302, and the loss of the second prediction layer as an example, the training process brings the loss function to converge, thus completing the training of the neural network model. Further details are omitted.

[0078] The target detection device provided by the present invention is described below. The target detection device described below and the target detection method described above can be referred to in correspondence.

[0079] The target detection device provided by this invention, such as Figure 5 As shown, it includes:

[0080] The target detection unit 51 is used to acquire the image to be detected and input it into the pre-trained target detection model to obtain the target detection result of the image to be detected output by the detection model;

[0081] The target detection model performs target detection on the image to be detected based on a preset first category of objects and a preset second category of objects, and merges the detection results of the first category of objects and the second category of objects to obtain the target detection result of the image to be detected.

[0082] Figure 6 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 6As shown, the electronic device may include a processor 610, a communication interface 620, a memory 630, and a communication bus 640, wherein the processor 610, the communication interface 620, and the memory 630 communicate with each other through the communication bus 640. The processor 610 can call logical instructions in the memory 630 to execute a target detection method, which includes: acquiring an image to be detected and inputting it into a pre-trained target detection model to obtain a target detection result output by the detection model; the target detection model performs target detection on the image to be detected based on a preset first category of objects and a preset second category of objects, and merges the detection results of the first category of objects and the second category of objects to obtain the target detection result of the image to be detected.

[0083] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0084] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the target detection method provided by the above methods. The method includes: acquiring an image to be detected and inputting it into a pre-trained target detection model to obtain a target detection result output by the detection model of the image to be detected; the target detection model performs target detection on the image to be detected based on a preset first category of objects and a preset second category of objects, and merges the detection results of the first category of objects and the second category of objects to obtain the target detection result of the image to be detected.

[0085] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the target detection method provided by the above methods. The method includes: acquiring an image to be detected and inputting it into a pre-trained target detection model to obtain a target detection result output by the detection model of the image to be detected; the target detection model performs target detection on the image to be detected based on a preset first category of objects and a preset second category of objects, and merges the detection results of the first category of objects and the second category of objects to obtain the target detection result of the image to be detected.

[0086] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0087] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0088] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A target detection method, characterized in that, include: The image to be detected is acquired and input into a pre-trained target detection model to obtain the target detection result output by the detection model of the image to be detected; The target detection model performs target detection on the image to be detected based on a preset first category of objects and a preset second category of objects, and merges the detection results of the first category of objects and the second category of objects to obtain the target detection result of the image to be detected. The target detection model includes a first detection module and a second detection module. The first detection module is used to obtain the target detection results of a preset first category of objects in the image to be detected, and the second detection module is used to obtain the target detection results of a preset second category of objects in the image to be detected. Wherein, the detection accuracy of the first category of objects in the preset classification and detection model is higher than the preset accuracy threshold, and the detection accuracy of the second category of objects in the preset classification and detection model is lower than the preset accuracy threshold. The first detection module includes: The first feature extraction layer is used to extract the primary features of the image to be detected; The first prediction layer is used to obtain the target detection result of the first category of objects based on the primary features. The second detection module includes: The second feature extraction layer is used to extract aggregate features of the input image to be detected based on the image to be detected and the primary features output by the first detection module. The second prediction layer is used to obtain the target detection results of the second category of objects based on the aggregated features.

2. The target detection method according to claim 1, characterized in that, The first feature extraction layer includes: The backbone network is used to extract multi-level features from the image to be detected; The feature fusion network is used to obtain primary features of the image to be detected based on the multi-level features extracted by the backbone network.

3. The target detection method according to claim 1, characterized in that, The second feature extraction layer includes: A region candidate network is used to obtain the target box prediction results of the second category of objects based on the primary features extracted by the first detection module. A feature aggregation network is used to obtain aggregated features based on the target bounding box prediction results and the image to be detected.

4. The target detection method according to claim 1, characterized in that, The second feature extraction layer includes: A region candidate network is used to obtain the target box prediction results of the second category of objects based on the primary features extracted by the first detection module; The original image extraction network is used to obtain the original image features based on the target bounding box prediction results and the image to be detected. A feature aggregation network is used to obtain aggregated features based on the target bounding box prediction results and the features of the original image.

5. A target detection device, characterized in that, include: The target detection unit is used to acquire the image to be detected and input it into a pre-trained target detection model to obtain the target detection result of the image to be detected output by the detection model; The target detection model performs target detection on the image to be detected based on a preset first category of objects and a preset second category of objects, and merges the detection results of the first category of objects and the second category of objects to obtain the target detection result of the image to be detected. The target detection model includes a first detection module and a second detection module. The first detection module is used to obtain the target detection results of a preset first category of objects in the image to be detected, and the second detection module is used to obtain the target detection results of a preset second category of objects in the image to be detected. Wherein, the detection accuracy of the first category of objects in the preset classification and detection model is higher than the preset accuracy threshold, and the detection accuracy of the second category of objects in the preset classification and detection model is lower than the preset accuracy threshold. The first detection module includes: The first feature extraction layer is used to extract the primary features of the image to be detected; The first prediction layer is used to obtain the target detection result of the first category of objects based on the primary features. The second detection module includes: The second feature extraction layer is used to extract aggregate features of the input image to be detected based on the image to be detected and the primary features output by the first detection module. The second prediction layer is used to obtain the target detection results of the second category of objects based on the aggregated features.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the target detection method as described in any one of claims 1 to 4.

Images