Target object detection method and device, electronic equipment and storage medium

Abstract

The application provides a target object detection method and device, electronic equipment and a computer readable storage medium. The target object detection method comprises: acquiring a state image of a preset area; performing feature extraction processing according to the state image to obtain image features of the state image; performing prediction processing according to the image features to obtain a first confidence degree of a preset first object category and a second confidence degree of a preset second object category respectively existing in the preset area; and determining whether a target object exists in the preset area according to the first confidence degree and the second confidence degree. In the application, the detection accuracy of the target object can be improved to a certain extent.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, in particular to a target object detection method and device, electronic equipment and computer readable storage medium. BACKGROUND

[0002] With the rapid development of computer vision technology, the application of computer vision technology is more and more widely, for example, whether there is a target object in a specific scene area can be detected based on the image of the specific scene area. Accurate detection of target objects is of positive significance for certain application scenarios, such as detecting whether there is a fast delivery falling in a logistics transfer field.

[0003] In the prior art, a detection model directly classifies (whether there is a target object, whether there is no target object) according to the image features of a specific scene area image to determine whether there is a target object in the image, and then determine whether there is a target object in the specific scene area.

[0004] However, the target object is sometimes single and sometimes multiple stacked, and because the placement of the target object is irregular, the image features of different images containing the target object are quite different, and direct classification based on image features will result in low detection accuracy of the target object. SUMMARY

[0005] The present application provides a target object detection method, device, electronic equipment and computer readable storage medium, aiming at solving the problem of low detection accuracy of the existing target object detection method when detecting irregularly placed target objects.

[0006] In a first aspect, the present application provides a target object detection method, the method comprising:

[0007] obtaining a state image of a preset area;

[0008] performing feature extraction processing according to the state image to obtain image features of the state image;

[0009] performing prediction processing according to the image features to obtain a first confidence degree of a preset first object category and a second confidence degree of a preset second object category of the preset area existing object;

[0010] determining whether the preset area exists a target object according to the first confidence degree and the second confidence degree.

[0011] In a second aspect, the present application provides a target object detection device, the target object detection device comprising:

[0012] an acquisition unit configured to obtain a state image of a preset area;

[0013] extracting a feature of the state image according to the state image to obtain an image feature of the state image;

[0014] predicting, by a prediction unit, according to the image feature, to obtain a first confidence degree of a preset first object category and a second confidence degree of a preset second object category of the object existing in the preset region;

[0015] determining, by a determination unit, according to the first confidence degree and the second confidence degree, whether the target object exists in the preset region.

[0016] In a possible implementation of the present application, before the step of determining whether the target object exists in the preset region according to the first confidence degree and the second confidence degree, the prediction unit is specifically configured to:

[0017] performing prediction processing according to the image feature to obtain a third confidence degree of the object existing in the preset region being a preset target object category, wherein the first object category and the second object category are subordinated to the target object category.

[0018] In a possible implementation of the present application, the determination unit is specifically configured to:

[0019] determine, according to the first confidence degree, the second confidence degree and the third confidence degree, whether the target object exists in the preset region.

[0020] In a possible implementation of the present application, the determination unit is specifically configured to:

[0021] obtain, from the first confidence degree, the second confidence degree and the third confidence degree, a confidence degree with the largest value as a target confidence degree of the target object existing in the preset region.

[0022] detect whether the target confidence degree is greater than a first preset threshold;

[0023] when it is detected that the target confidence degree is greater than the first preset threshold, determine that the target object exists in the preset region;

[0024] when it is detected that the target confidence degree is less than or equal to the first preset threshold, determine that the target object does not exist in the preset region.

[0025] In a possible implementation of the present application, the prediction unit is specifically configured to:

[0026] invoke a first prediction subnetwork in the first detection network after training to perform prediction processing according to the image feature to obtain the first confidence degree and the second confidence degree;

[0027] invoke a second prediction sub-network in the first detection network after training to perform prediction processing according to the image feature, to obtain the third confidence, wherein the first prediction sub-network and the second prediction sub-network are parallel branches.

[0028] In a possible implementation of the present application, the determination unit is specifically configured to:

[0029] perform prediction processing according to the first confidence and the second confidence, to obtain a fourth confidence that the object in the preset region is a preset target object category;

[0030] detect whether the fourth confidence is greater than a second preset threshold;

[0031] when it is detected that the fourth confidence is greater than the second preset threshold, determine that the preset region contains a target object;

[0032] when it is detected that the fourth confidence is less than or equal to the second preset threshold, determine that the preset region does not contain a target object.

[0033] In a possible implementation of the present application, the prediction unit is specifically configured to:

[0034] invoke a third prediction sub-network in the second detection network after training to perform prediction processing according to the image feature, to obtain the first confidence and the second confidence;

[0035] invoke a fourth prediction sub-network in the second detection network after training to perform prediction processing according to the first confidence and the second confidence, to obtain the fourth confidence, wherein the third prediction sub-network and the fourth prediction sub-network are cascaded branches.

[0036] In a possible implementation of the present application, the target object detection apparatus further includes a training unit, and the training unit is specifically configured to:

[0037] obtain a training data set, wherein the training data set includes a first sample image of the first object category and a second sample image of the second object category;

[0038] obtain a data difference value between the first sample image and the second sample image;

[0039] when the data difference value is greater than a preset quantity threshold, perform augmentation on a sample image of a target sample category to obtain an updated training data set, wherein the target sample category refers to a category with the least number of sample images in the first object category and the second object category;

[0040] Train the first preset network according to the updated training data set, to obtain the second detection network.

[0041] In a possible implementation of the present application, the target object detection apparatus further comprises an alarm unit, which is specifically configured to:

[0042] When it is determined that the target object exists in the preset area, output abnormal alarm information.

[0043] In a third aspect, the present application also provides an electronic device, which comprises a processor and a memory, and the memory stores a computer program, and the processor invokes the computer program in the memory to execute the steps in any of the target object detection methods provided in the present application.

[0044] In a fourth aspect, the present application also provides a computer readable storage medium, which stores a computer program, and the computer program is loaded by a processor to execute the steps in the target object detection method.

[0045] The present application detects the confidence of different placement states (of the target object) with large differences respectively, i.e., respectively predicts the first confidence that the preset area exists the object of the preset first object category and the second confidence that the preset area exists the object of the preset second object category, and then determines whether the target object exists in the preset area based on the first confidence and the second confidence. The present application can avoid the binary classification (of the existence of the target object or the non-existence of the target object) based on the image features directly, and further avoid the problem that the image features of different placement states of the target object are quite different, and the binary classification model is difficult to accurately determine whether the target object exists, thereby improving the detection accuracy of the target object. BRIEF DESCRIPTION OF DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0047] Figure 1 is a scene schematic diagram of the target object detection system provided by the embodiments of the present application;

[0048] Figure 2 is a flowchart of the target object detection method provided by the embodiments of the present application;

[0049] Figure 3 is a relationship schematic diagram of the image features and the first confidence and the second confidence;

[0050] Figure 4 This is a schematic diagram illustrating another relationship between image features and the first and second confidence levels, respectively.

[0051] Figure 5 This is a schematic diagram of a structure of the first detection network provided in the embodiments of this application;

[0052] Figure 6 This is a schematic diagram of a second detection network provided in the embodiments of this application;

[0053] Figure 7 This is a schematic diagram of an embodiment of the target object detection device provided in this application.

[0054] Figure 8 This is a schematic diagram of an embodiment of the electronic device provided in this application. Detailed Implementation

[0055] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0056] In the description of the embodiments of this application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, features defined with "first" and "second" may explicitly or implicitly include one or more of the stated features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0057] To enable any person skilled in the art to implement and use this application, the following description is provided. In this description, details are set forth for purposes of explanation. It should be understood that those skilled in the art will recognize that this application can be implemented without using these specific details. In other instances, well-known processes will not be described in detail to avoid obscuring the description of the embodiments of this application with unnecessary detail. Therefore, this application is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in the embodiments of this application.

[0058] This application provides a target object detection method, apparatus, electronic device, and computer-readable storage medium. The target object detection apparatus can be integrated into an electronic device, which may be a server or a terminal, etc.

[0059] Before introducing the embodiments of the present application, the related content about the application background of the embodiments of the present application is introduced first.

[0060] Computer vision technology is applied in more and more fields. For example, in the logistics field, there are a large number of express items in the transfer field and the like. In order to count the number of express items, or to prevent express items from falling off or being lost, a computer vision method is often used to detect express items to play a real-time monitoring role.

[0061] However, falling off or being lost is sometimes single placement, and sometimes multiple stacking placement. At the same time, a (target object exists, target object does not exist) binary classification network is trained based on a single express item sample image and a pile of express items as training data sets. Since there is a large difference between a single express item and a pile of express items, the precision of the binary classification network for express item detection is relatively low after being trained based on a single express item sample image and a pile of express items as training data sets.

[0062] Based on the above defects existing in the prior art, the target object detection method provided in the embodiments of the present application at least partially overcomes the defects existing in the prior art.

[0063] The execution subject of the target object detection method provided in the embodiments of the present application can be a target object detection device provided in the embodiments of the present application, or a server device, a physical host or a user equipment (User Equipment, UE) and the like of different types of electronic devices integrated with the target object detection device. The target object detection device can be realized in the form of hardware or software. The UE can be a terminal device such as a smart phone, a tablet computer, a notebook computer, a palm computer, a desktop computer or a personal digital assistant (Personal Digital Assistant, PDA) and the like.

[0064] The electronic device can run in a separate mode or in a device cluster mode. By applying the target object detection method provided in the embodiments of the present application, the detection precision of the target object can be improved to a certain extent.

[0065] Referring to Figure 1 , Figure 1is a scene schematic diagram of a target object detection system provided by an embodiment of the present application. The target object detection system can include an electronic device 100, and the electronic device 100 is integrated with a target object detection apparatus. For example, the electronic device can acquire a state image of a preset area; perform feature extraction processing according to the state image to obtain image features of the state image; perform prediction processing according to the image features to obtain a first confidence degree that the preset area exists a first object class and a second confidence degree that the preset area exists a second object class; and determine whether the preset area exists a target object according to the first confidence degree and the second confidence degree.

[0066] In addition, as shown in Figure 1 The target object detection system can further include a memory 200 configured to store data, such as a state image of a preset area.

[0067] It should be noted that Figure 1 The scene schematic diagram of the target object detection system shown is only an example, and the target object detection system and the scene described in the embodiments of the present application are used to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art can know that, as the target object detection system evolves and new business scenarios appear, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0068] Next, a target object detection method provided by an embodiment of the present application will be introduced. In the embodiments of the present application, an electronic device is taken as an execution subject, and the execution subject will be omitted in subsequent method embodiments for simplification and convenience of description.

[0069] Referring to Figure 2 , Figure 2 is a flowchart of a target object detection method provided by an embodiment of the present application. It should be noted that although a logical sequence is shown in the flowchart, in some cases, the steps shown or described can be performed in an order different from that shown. The target object detection method includes steps S10-S40, wherein:

[0070] S10, acquiring a state image of a preset area.

[0071] The preset area can be adaptively changed according to specific application scenarios. For example, in order to detect whether a fast delivery falls in a logistics transfer field, the preset area refers to the transfer field. For another example, in order to detect whether there is a person on a bus station platform, the preset area refers to the bus station platform.

[0072] The preset area can be provided with a camera, a field of view range of the camera can cover the preset area, the camera can be used to collect a state image (hereinafter referred to as a “state image”) of the preset area, so as to be used for subsequent analysis and determination of whether a target object exists in the preset area based on the state image.

[0073] Specifically, in actual application, the electronic device applying the target object detection method provided in the application can directly include a camera of the preset area in hardware (the camera is mainly used to collect an image containing the preset area), and store the image obtained by the camera locally, which can be directly read in the electronic device; or the electronic device can also establish a network connection with the camera, and obtain the image obtained by the camera online according to the network connection; or the electronic device can also read the image obtained by the camera from a related storage medium storing the image obtained by the camera, and the specific acquisition mode is not limited here.

[0074] The camera can shoot images according to a preset shooting mode, for example, a shooting height, a shooting direction or a shooting distance can be set, and the specific shooting mode can be adjusted according to the camera itself, and the specific shooting mode is not limited here. The multiple frames of images obtained by the camera can form a video through a time line.

[0075] S20, performing feature extraction processing on the state image to obtain an image feature of the state image.

[0076] The image feature refers to an image space feature obtained after the feature extraction processing on the state image.

[0077] In some embodiments, the first detection network after training can be used to perform feature extraction on the state image to obtain the image feature of the state image. The process of feature extraction by the first detection network will be described in detail hereinafter, and will not be repeated here.

[0078] In some embodiments, the second detection network after training can be used to perform feature extraction on the state image to obtain the image feature of the state image. The process of feature extraction by the second detection network will be described in detail hereinafter, and will not be repeated here.

[0079] S30, performing prediction processing according to the image feature to obtain a first confidence degree that the object existing in the preset area is a preset first object category and a second confidence degree that the object existing in the preset area is a preset second object category.

[0080] The first object category and the second object category are respectively categories indicating two different placement states of the target object. For example, the first object category is "single target object", and the second object category is "a pile of target objects". For another example, the first object category is "target object with a number less than 3", and the second object category is "target object with a number greater than or equal to 3".

[0081] In the embodiments of the present application, the purpose of predicting the first confidence that the object in the preset area is the preset first object category and the second confidence that the object in the preset area is the preset second object category is to: for irregular placement of the target object, the confidence of different placement states (in which the target object is) with large differences is detected, direct binary classification (whether the target object exists) based on image features is avoided, and then the problem that the image features of different placement states of the target object are greatly different and the binary classification model is difficult to accurately determine whether the target object exists is avoided, thereby improving the detection accuracy of the target object.

[0082] The first confidence refers to the confidence that the object in the preset area is the preset first object category. For example, if the first object category is "single express delivery", the first confidence refers to the confidence that "single express delivery" exists in the preset area.

[0083] The second confidence refers to the confidence that the object in the preset area is the preset second object category. For example, if the second object category is "a pile of express deliveries", the second confidence refers to the confidence that "a pile of express deliveries" exists in the preset area.

[0084] In some embodiments, on the one hand, the first confidence that the object in the preset area is the preset first object category is obtained by performing classification prediction processing according to image features. On the other hand, the first confidence that the object in the preset area is the preset second object category is obtained by performing classification prediction processing according to image features. For example, Figure 3 as shown in Figure 3 is a schematic diagram of a relationship between image features and the first confidence and the second confidence. That is, after performing feature extraction once based on the state image, the obtained image features are used for first confidence prediction and second confidence prediction.

[0085] In some embodiments, the image features can include first image features and second image features. At this time, on the one hand, the first confidence that the object in the preset area is the preset first object category is obtained by performing classification prediction processing according to the first image features. On the other hand, the first confidence that the object in the preset area is the preset second object category is obtained by performing classification prediction processing according to the second image features. Please refer to Figure 4 , Figure 4 is another schematic diagram of a relationship between image features and the first confidence and the second confidence. For example, Figure 4The first image feature and the second image feature are obtained based on the state image at different layers, respectively, as shown in (a) and (b), the obtained first image feature is used for first confidence prediction, and the obtained second image feature is used for second confidence prediction.

[0086] S40, determining whether the preset area has a target object according to the first confidence and the second confidence.

[0087] In some embodiments, step S40 can specifically include: comparing whether the maximum value in the first confidence and the second confidence is greater than a preset confidence threshold. When the maximum value in the first confidence and the second confidence is greater than the preset confidence threshold, it is determined that the preset area has a target object. When the maximum value in the first confidence and the second confidence is less than or equal to the preset confidence threshold, it is determined that the preset area does not have a target object.

[0088] For example, in order to detect whether a parcel has fallen in the site A of a logistics transfer site, according to the state image of the site A, it is detected that the first confidence of the site A falling “a single parcel” is 0.5, and the second confidence of the site A falling “a pile of parcels” is 0.8. If the maximum value in the first confidence 0.5 and the second confidence 0.8 is greater than the preset confidence threshold 0.75, it can be determined that the preset area has a target object. If the maximum value in the first confidence 0.5 and the second confidence 0.8 is less than the preset confidence threshold 0.85, it can be determined that the preset area does not have a target object. Here, the preset confidence threshold is only an example, and can be set according to actual needs, not limited thereto.

[0089] In some embodiments, the image feature can be further used for prediction processing to obtain a third confidence that the object in the preset area is a preset target object category. Then, whether the maximum value in the first confidence, the second confidence and the third confidence is greater than a preset confidence threshold is compared. When the maximum value in the first confidence, the second confidence and the third confidence is greater than the preset confidence threshold, it is determined that the preset area has a target object. When the maximum value in the first confidence, the second confidence and the third confidence is less than or equal to the preset confidence threshold, it is determined that the preset area does not have a target object. The specific implementation of determining whether the preset area has a target object will be described in detail and exemplified later, which is simplified here.

[0090] In some embodiments, the first confidence level and the second confidence level can be further used as a basis for prediction processing to obtain a fourth confidence level that the object in the preset area is of the preset target object category. Then, a comparison is made as to whether the maximum value among the first confidence level, the second confidence level and the fourth confidence level is greater than a preset confidence threshold. When the maximum value among the first confidence level, the second confidence level and the fourth confidence level is greater than the preset confidence threshold, it is determined that the target object exists in the preset area. When the maximum value among the first confidence level, the second confidence level and the fourth confidence level is less than or equal to the preset confidence threshold, it is determined that the target object does not exist in the preset area.

[0091] In some embodiments, the first confidence level and the second confidence level can be further used as a basis for prediction processing to obtain a fourth confidence level that the object in the preset area is of the preset target object category. Then, a comparison is made as to whether the maximum value among the first confidence level, the second confidence level and the fourth confidence level is greater than a preset confidence threshold. When the maximum value among the first confidence level, the second confidence level and the fourth confidence level is greater than the preset confidence threshold, it is determined that the target object exists in the preset area. When the maximum value among the first confidence level, the second confidence level and the fourth confidence level is less than or equal to the preset confidence threshold, it is determined that the target object does not exist in the preset area.

[0092] As can be seen from the above, for the irregularity of the placement of the target object, the confidence levels of different placement states (in which the target object is in) with large differences are detected respectively, i.e., the first confidence level that the object in the preset area is of the preset first object category, and the second confidence level that the object in the preset area is of the preset second object category are predicted respectively; and then, whether the target object exists in the preset area is determined based on the first confidence level and the second confidence level. This can avoid direct binary classification (whether the target object exists) based on image features, and further avoid the problem that the image features of different placement states of the target object are quite different, and the binary classification model is difficult to accurately determine whether the target object exists, thereby improving the detection accuracy of the target object.

[0093] In order to improve the detection accuracy of the target object, in some embodiments, before step S40, the prediction processing can be further included, i.e., according to the image features, a third confidence level that the object in the preset area is of the preset target object category is obtained.

[0094] The target object category refers to the category of the target object. For example, in order to detect whether the express exists in the preset area, the target object category can be “express”. For another example, in order to detect whether the person exists in the preset area, the target object category can be “person”.

[0095] The third confidence level refers to the confidence level that the object in the preset area is of the preset target object category obtained according to the image features of the state image, and specifically refers to the confidence level that the target object exists in the preset area.

[0096] In this context, both the first object category and the second object category belong to the target object category.

[0097] For example, the target object category is "express delivery", the first object category is "single express delivery", and the second object category is "a pile of express delivery". Both "single express delivery" and "a pile of express delivery" belong to "express delivery".

[0098] At this point, step S40 may specifically include: determining whether a target object exists in the preset area based on the first confidence level, the second confidence level, and the third confidence level.

[0099] As can be seen from the above, the presence of an object in the preset region is evaluated based on the first confidence level of the first object category, the second confidence level of the second object category, and the third confidence level of the target object category, respectively. This evaluation considers both cases of irregular placement and cases without considering irregular placement, thus improving the comprehensiveness of the predicted features and consequently increasing the detection accuracy of the target object.

[0100] Specifically, a first detection network after deep learning can be used to extract features from the state image to obtain the image features of the state image and predict the first confidence level, the second confidence level, and the third confidence level.

[0101] That is, step S20 may specifically include: calling the feature extraction layer in the first detection network after training, performing feature extraction processing based on the state image, and obtaining the image features of the state image.

[0102] Step S30 may specifically include: calling the first prediction subnetwork in the trained first detection network, performing prediction processing based on image features, and obtaining the first confidence score and the second confidence score.

[0103] The phrase "performing prediction processing based on the image features to obtain a third confidence level that the object in the preset region belongs to a preset target object category" can specifically include: calling the second prediction subnetwork in the trained first detection network to perform prediction processing based on the image features and obtain the third confidence level. The first prediction subnetwork and the second prediction subnetwork are parallel branches.

[0104] like Figure 5 As shown, Figure 5 This is a schematic diagram of a first detection network provided in an embodiment of this application. The first detection network may include a feature extraction layer and a prediction layer. The prediction layer may further include a first prediction subnetwork and a second prediction subnetwork.

[0105] The feature extraction layer is configured to output image features of the state image (i.e., the image of the preset region) according to the state image (i.e., the image of the preset region). The feature extraction layer takes the state image (i.e., the image of the preset region) as input, and performs one or more operations including but not limited to convolution, pooling, etc. on the state image to achieve feature extraction of the state image to obtain the image features of the state image. The feature extraction layer can be a convolutional neural network (CNN), for example, a CNN network obtained by removing the fully connected layer of the classification network ResNet50 can be directly used as the feature extraction layer.

[0106] The first prediction sub-network is configured to output a first confidence and a second confidence according to the image features of the state image. The first prediction sub-network takes the image features of the state image as input, and respectively performs regression classification prediction processing according to the image features of the state image to determine a first confidence that the object existing in the preset region is a first preset object category and a second confidence that the object existing in the preset region is a second preset object category.

[0107] The second prediction sub-network is configured to output a third confidence according to the image features of the state image. The second prediction sub-network takes the image features of the state image as input, and performs regression classification prediction processing according to the image features of the state image to determine a third confidence that the object existing in the preset region is a preset target object category.

[0108] As can be seen from the above, since the first detection network learns the feature relationship between the image features and the first confidence, the second confidence, and the third confidence after being trained. By setting the first detection network (the first prediction sub-network and the second prediction sub-network are parallel branches) to predict the first confidence, the second confidence, and the third confidence, the reliability of the first confidence, the second confidence, and the third confidence is improved to a certain extent.

[0109] In some embodiments, the first detection network is trained by the second preset network through steps a1-a5 as follows:

[0110] a1, obtaining a sample image of a target object.

[0111] The sample image is labeled with corresponding category information, regression box information, and confidence of the labeled category information. The first category information includes a first object category and a second object category. The labeled first object category and the second object category both belong to the target object category.

[0112] For example, in order to enable the trained first detection network to detect express delivery. Data labeling is performed on the pictures of the transfer field, and all express deliveries appearing in the pictures are labeled. Among them, a single express delivery is labeled as a class, marked as class 0; the express deliveries stacked together are divided into another class, marked as class 1, as long as the express deliveries in the picture have a connection place, they are classified as class 1; the labeled information is converted into the required data format to form the express delivery binary classification data set.

[0113] a2, calling the feature extraction layer in the second preset network to perform feature extraction processing on the sample image to obtain sample features of the sample image.

[0114] Among them, the second preset network can be an open source network model that can be used for detection tasks, such as YOLOv1 network, YOLOv2 network, YOLOv3 network, SSD network and RetinaNet network, etc. (Among them, the feature extraction layer of the open source network model is the feature extraction layer in the second preset network; the prediction layer of the open source network model is the first prediction sub-network in the second preset network), and an additional prediction sub-network (i.e. the second prediction sub-network) is added on this basis; wherein the first prediction sub-network and the second prediction sub-network are parallel branches. The network structure and function implementation of the second preset network are similar to those of the first detection network, and specific reference can be made to the description of the network structure and function implementation of the first detection network, which will not be repeated here.

[0115] a3, calling the prediction layer in the second preset network to perform classification prediction processing according to the sample features to obtain the confidence that the object existing in the sample image is the first object class, the confidence that the object existing in the sample image is the second object class, and the confidence that the object existing in the sample image is the target object class.

[0116] a4, according to the confidence that the object existing in the sample image is the first object class, the confidence that the object existing in the sample image is the second object class, and the confidence that the object existing in the sample image is the target object class, and the corresponding class information, the regression box information, and the confidence of the labeled class information of the sample image, determine the total loss value of the second preset network.

[0117] For example, the total loss value of the second preset network can be determined according to the following formula (1):

[0118] L = L obj + L cls + L bbox + L cls_1 Formula (1)

[0119] Among them, L is the total loss value, L obja confidence loss (determined by a confidence that an object exists in a sample image predicted by the first prediction sub-network is a first object class, a confidence that an object exists in a sample image predicted by the first prediction sub-network is a second object class, a confidence that an object exists in a sample image predicted by the first prediction sub-network is a target object class, and a confidence that the labeled class information is correct), L cls a classification loss of the first prediction sub-network (determined by the labeled class information and a classification result of the first prediction sub-network), L bbox a regression loss (determined by the labeled regression box information and a regression result of the second preset network), L cls_1 a classification loss of the second prediction sub-network (determined by the labeled class information and a classification result of the second prediction sub-network)

[0120] Further, in order to make the loss of the second prediction sub-network be added to the first detection network with different weights, when the classification tendency of the target object class is not enough, the classification tendency of the target object class of the first detection network is improved. That is, the total loss value of the training of the second preset network can be determined according to the following formula (2):

[0121] L = L obj + L cls + L bbox + aL cls_1 Formula (2)

[0122] wherein, L is the total loss value, L obj is the confidence loss, L cls is the classification loss of the first prediction sub-network, L bbox is the regression loss, L cls_1 is the classification loss of the second prediction sub-network, and a is a weight parameter.

[0123] a5, updating the model parameters of the second preset network according to the total loss value of the training until the second preset network converges, and taking the second preset network as the trained first detection network.

[0124] In some embodiments, the above-mentioned "determining whether the preset region exists a target object according to the first confidence, the second confidence and the third confidence" can specifically include: obtaining a confidence with the largest value from the first confidence, the second confidence and the third confidence as a target confidence that the preset region exists a target object; detecting whether the target confidence is greater than a first preset threshold; when it is detected that the target confidence is greater than the first preset threshold, determining that the preset region exists a target object; and when it is detected that the target confidence is less than or equal to the first preset threshold, determining that the preset region does not exist a target object.

[0125] For example, in order to detect whether a parcel falls in the site A of the logistics transfer site, according to the state image of the site A, a first confidence degree of the site A falling "a single parcel" is 0.5, a second confidence degree of the site A falling "a pile of parcels" is 0.8, and a third confidence degree of the site A falling "a parcel" is 0.7. If the maximum value (i.e. the target confidence degree) of the first confidence degree 0.5, the second confidence degree 0.8 and the third confidence degree 0.7 is greater than the first preset threshold 0.75, it can be determined that the target object exists in the preset area.

[0126] Here, the first preset threshold is only an example, and can be set according to actual needs, and is not limited thereto.

[0127] The target object category refers to the category of the target object. For example, in order to detect whether a parcel falls in the site A of the logistics transfer site, according to the state image of the site A, a first confidence degree of the site A falling "a single parcel" is 0.5, a second confidence degree of the site A falling "a pile of parcels" is 0.8, and a third confidence degree of the site A falling "a parcel" is 0.7. If the maximum value (i.e. the target confidence degree) of the first confidence degree 0.5, the second confidence degree 0.8 and the third confidence degree 0.7 is greater than the first preset threshold 0.75, it can be determined that the target object exists in the preset area.

[0128] The third confidence degree refers to the confidence degree that the object existing in the preset area is the preset target object category, and specifically refers to the confidence degree that the target object exists in the preset area.

[0129] The target confidence degree refers to the maximum value of the first confidence degree, the second confidence degree and the third confidence degree, and specifically refers to the confidence degree that the target object exists in the preset area.

[0130] The first object category and the second object category both belong to the target object category.

[0131] For example, the target object category is "parcel", the first object category is "single parcel", and the second object category is "a pile of parcels", and "single parcel" and "a pile of parcels" both belong to "parcel".

[0132] From the above, it can be seen that since the object existing in the preset area is the preset first object category, the object existing in the preset area is the preset second object category, and the object existing in the preset area is the preset target object category, the target object existing in the preset area can be reflected, and whether the maximum value of the first confidence degree, the second confidence degree and the third confidence degree is greater than the first preset threshold is determined to determine whether the target object exists in the preset area, which improves the accuracy of the determination result to a certain extent.

[0133] In some embodiments, step S40 can specifically include: performing a prediction process according to the first confidence and the second confidence to obtain a fourth confidence that the object existing in the preset area is a preset target object category; detecting whether the fourth confidence is greater than a second preset threshold; when it is detected that the fourth confidence is greater than the second preset threshold, determining that the target object exists in the preset area; and when it is detected that the fourth confidence is less than or equal to the second preset threshold, determining that the target object does not exist in the preset area.

[0134] For example, in order to detect whether a parcel has fallen in the site B of the logistics transfer field. First, according to the state image of the site B, it is detected that the first confidence that the site B falls “a single parcel” is 0.5, and the second confidence that the site B falls “a pile of parcels” is 0.8. Then, the first confidence 0.5 and the second confidence 0.8 are taken as features to make a prediction, and the fourth confidence that the site B falls “a parcel” is obtained as 0.7. Finally, it is detected whether the fourth confidence 0.7 is greater than a second preset threshold. If it is detected that the fourth confidence 0.7 is greater than the second preset threshold (such as 0.6), it can be determined that the target object exists in the preset area. If it is detected that the fourth confidence 0.7 is less than or equal to the second preset threshold (such as 0.8), it can be determined that the target object does not exist in the preset area.

[0135] As can be seen from the above, since the fourth confidence is the confidence that the object existing in the preset area is the preset target object category, when the fourth confidence is greater than the second preset threshold, it is determined that the target object exists in the preset area, which can improve the accuracy of the determination result to a certain extent.

[0136] Specifically, the second detection network after deep learning can be used to extract features of the state image to obtain the image features of the state image, predict the first confidence, the second confidence and the fourth confidence.

[0137] That is, step S20 can specifically include: calling a feature extraction layer in the second detection network after training, performing feature extraction processing according to the state image to obtain the image features of the state image.

[0138] Step S30 can specifically include: calling a third prediction sub-network in the second detection network after training, performing a prediction process according to the image features to obtain the first confidence and the second confidence.

[0139] The "performing prediction processing according to the first confidence and the second confidence to obtain a fourth confidence that the object existing in the preset region is a preset target object category" can specifically include: calling a fourth prediction subnetwork in the trained second detection network, performing prediction processing according to the first confidence and the second confidence to obtain the fourth confidence. The third prediction subnetwork and the fourth prediction subnetwork are cascade branches.

[0140] As shown in Figure 6 , Figure 6 is a structural schematic diagram of a second detection network provided in an embodiment of the present application. The second detection network can include a feature extraction layer and a prediction layer. The prediction layer can further include a third prediction subnetwork and a fourth prediction subnetwork.

[0141] The feature extraction layer is configured to output image features of a state image (i.e., an image of a preset region) according to the state image. The feature extraction layer takes the state image as input and performs one or more operations including but not limited to convolution and pooling on the state image to extract features of the state image to obtain the image features of the state image. The feature extraction layer can be a convolutional neural network (CNN), for example, a CNN network obtained by removing the fully connected layer of the classification network ResNet50.

[0142] The third prediction subnetwork is configured to output a first confidence and a second confidence according to the image features of the state image. The first prediction subnetwork takes the image features of the state image as input and performs regression classification prediction processing according to the image features of the state image to determine a first confidence that the object existing in the preset region is a preset first object category and a second confidence that the object existing in the preset region is a preset second object category.

[0143] The fourth prediction subnetwork is configured to output a fourth confidence according to the first confidence and the second confidence. The second prediction subnetwork takes the first confidence and the second confidence as feature input and performs regression classification prediction processing according to the first confidence and the second confidence to determine a fourth confidence that the object existing in the preset region is a preset target object category.

[0144] As can be seen from the above, since the second detection network learns the feature relationship between the image features and the first confidence and the second confidence and the feature relationship between the first confidence and the second confidence and the fourth confidence after being trained. By setting the second detection network (the third prediction subnetwork and the fourth prediction subnetwork are cascade branches) to predict the first confidence, the second confidence, and the fourth confidence, the reliability of the first confidence, the second confidence, and the third confidence is improved to a certain extent.

[0145] In some embodiments, the second detection network is trained by the first preset network through steps b1-b5 as follows:

[0146] b1, obtaining a training data set.

[0147] The training data set includes first sample images of a first object category and second sample images of a second object category. The first sample images and the second sample images (referred to as sample images) are labeled with corresponding category information, regression box information, and confidence of the labeled category information. The first category information includes the first object category and the second object category. The labeled first object category and the second object category both belong to the target object category.

[0148] For example, in order to enable the trained second detection network to detect express delivery, data labeling is performed on pictures of the transfer site to label all express delivery in the pictures. Single express delivery is labeled as one category, marked as 0; express delivery stacked together is labeled as another category, marked as 1, as long as the express delivery in the picture has a connection place, it is classified as 1; the labeled information is converted into the required data format to form an express delivery two-classification data set.

[0149] b2, calling the feature extraction layer in the first preset network to perform feature extraction processing on the sample images (including the first sample images and the second sample images) to obtain sample features of the sample images.

[0150] The first preset network can be an open source network model that can be used for detection tasks, such as YOLOv1 network, YOLOv2 network, YOLOv3 network, SSD network, and RetinaNet network, etc. (wherein the feature extraction layer of the open source network model is the feature extraction layer in the first preset network; the prediction layer of the open source network model is the third prediction sub-network in the first preset network), and an additional prediction sub-network (i.e. the fourth prediction sub-network) is added on this basis; the third prediction sub-network and the fourth prediction sub-network are cascaded branches. The network structure and function implementation of the first preset network are similar to those of the second detection network, and specific descriptions can be referred to the network structure and function implementation of the second detection network, which will not be repeated here.

[0151] b3, calling the third prediction sub-network in the first preset network to perform classification prediction processing according to the sample features to obtain confidence of the first object category and confidence of the second object category existing in the sample images.

[0152] b4, calling a fourth prediction sub-network in the first preset network, performing prediction processing according to the confidence that the existing object in the sample image is the first object category and the confidence that the existing object in the sample image is the second object category, to obtain the confidence that the existing object in the sample image is the target object category.

[0153] b5, determining the total loss value of the training of the first preset network according to the confidence that the existing object in the sample image is the first object category, the confidence that the existing object in the sample image is the second object category, the confidence that the existing object in the sample image is the target object category, the corresponding category information, the regression box information and the confidence of the labeled category information of the sample image.

[0154] For example, the total loss value of the training of the first preset network can be determined according to the following formula (3):

[0155] L' = L' obj + L' cls + L' bbox + L' cls_1 Formula (3)

[0156] Wherein, L' is the total loss value of the training, L' obj is the confidence loss (determined by the confidence that the existing object in the sample image is the first object category, the confidence that the existing object in the sample image is the second object category, the confidence that the existing object in the sample image is the target object category, and the confidence of the labeled category information), L' cls is the classification loss of the third prediction sub-network (determined by the labeled category information and the classification result of the third prediction sub-network), L' bbox is the regression loss (determined by the labeled regression box information and the regression result of the first preset network), L' cls_1 is the classification loss of the fourth prediction sub-network (determined by the labeled category information and the classification result of the fourth prediction sub-network)

[0157] Further, in order to make the loss of the fourth prediction sub-network join the second detection network with different weights, when the classification tendency of the target object category is not enough, the classification tendency of the target object category of the second detection network is improved. That is, the total loss value of the training of the first preset network can be determined according to the following formula (4):

[0158] L' = L' obj + L' cls + L' bbox + aL' cls_1 Formula (4)

[0159] Wherein, L' is the total loss value of the training, L' obj is the confidence loss, L' cls is the classification loss of the third prediction sub-network, L' bboxFor regression loss, L' cls_1 Let α be the classification loss of the fourth prediction sub-network, and α be the weight parameter.

[0160] b6. Update the model parameters of the first preset network according to the total training loss value until the first preset network converges, and then use the first preset network as the trained second detection network.

[0161] Furthermore, to avoid a large difference in the number of samples between the first and second object categories (i.e., an unbalanced distribution of sample data), which could lead to a decrease in the accuracy of the second detection network in detecting the target object, some embodiments of this application may further expand the data for the smaller category.

[0162] Therefore, in some embodiments, the training dataset in step b1 can be obtained through the following steps c1 to c3, wherein:

[0163] c1. Obtain the training dataset.

[0164] The training dataset includes first sample images of the first object category and second sample images of the second object category.

[0165] c2. Obtain the data difference between the first sample image and the second sample image.

[0166] For example, if the number of first sample images is 500 and the number of second sample images is 200, then the data difference between the first sample images and the second sample images is: 500 - 200 = 300.

[0167] c3. When the data difference is greater than a preset threshold, the sample images of the target sample category are amplified to obtain an updated training dataset.

[0168] The target sample category refers to the category with the fewest sample images among the first object category and the second object category.

[0169] Specifically, when the data difference between the first sample image and the second sample image is greater than a preset threshold, the sample images of the target sample category need to be augmented. When the data difference between the first sample image and the second sample image is less than or equal to the preset threshold, since the imbalance between the sample data is low, it will not affect the detection accuracy of the trained second detection network for the target object, and there is no need to augment the sample images of the target sample category.

[0170] For example, the number of first sample images of the first object category is 900, and the number of second sample images of the second object category is 500. If the data difference (900-500=400) between the first sample images and the second sample images is greater than a preset number threshold (for example, 100), the sample images of the second object category need to be augmented.

[0171] At this time, further data augmentation can be performed based on the second sample images of the second object category until the number of samples of the first object category and the second object category is balanced (for example, the number of samples of the two categories is equal, or the difference between the number of samples of the two categories is less than a preset value).

[0172] Finally, the updated training data set is obtained, that is, all sample data before and after augmentation is used as the final training data set. The first preset network is trained according to the updated training data set, and the second detection network is obtained (the specific implementation of training the first preset network can refer to steps b2-b5 described above, and will not be described here).

[0173] The specific data augmentation method can include but is not limited to:

[0174] (1) First, obtain the label information of all position boxes of the second sample images of the second object category;

[0175] (2) According to the label information, the target object picture in the second sample image original picture is cropped;

[0176] (3) The cropped sample picture is subjected to scale transformation of enlargement and reduction and random angle rotation transformation;

[0177] (4) The transformed image is overlaid on the places without labels of other images to increase the number of labels.

[0178] It can be understood that the sample data augmentation method in steps c1-c3 is also applicable to the sample data in the training process of the first detection network in steps a1-a5.

[0179] From the above, it can be seen that when the data difference between the first sample images and the second sample images is greater than the preset number threshold, by augmenting the sample data of the category with the least number of sample images in the first object category and the second object category, the problem of unbalanced sample data of the two categories in the training data set can be avoided, thereby improving the accuracy of the second detection network in detecting target objects.

[0180] Further, in order to timely find out whether the target object exists in the preset area, so as to enable the manager to timely further dispose the target object in the preset area, in some embodiments of the present application, the target object detection method further comprises: when it is determined that the target object exists in the preset area, outputting abnormal alarm information.

[0181] For example, when the transfer site needs to be cleared, when it is detected that the express (i.e. the target object is the express) still exists in the transfer site (i.e. the preset area), an abnormal alarm sound of clearing the site, an abnormal alarm subtitle of clearing the site or an abnormal alarm light of clearing the site is output. So that the relevant workers or managers can clean up the missed express in the transfer site in time after receiving the abnormal alarm information, and complete the clearing work of the transfer site in time.

[0182] For another example, when the subway needs to be closed, when it is detected that there are still people (i.e. the target object is people) in the subway platform area (i.e. the preset area), an abnormal alarm sound, an abnormal alarm subtitle or an alarm light is output. So that the relevant workers or managers can prompt the passengers staying in the subway platform area in time after receiving the abnormal alarm information, and ensure that the subway can be closed normally.

[0183] From the above, it can be seen that when it is determined that the target object exists in the preset area, the abnormal alarm information is output, so that the relevant workers can timely find out the target object in the preset area, and ensure that the work under the specific scene can be executed normally.

[0184] Further, in order to detect the detection accuracy of the first detection network or the second detection network on the target object, in some embodiments of the present application, a way of evaluating the performance of the first detection network or the second detection network is provided. So that whether the detection accuracy performance of the first detection network or the second detection network meets the requirement can be evaluated, and when the detection accuracy performance does not meet the requirement, data can be further supplemented or the network can be further modified for training, so as to further improve the detection accuracy of the first detection network or the second detection network on the target object.

[0185] Specifically, taking the second detection network as an example, the network performance evaluation way is described.

[0186] For example, with reference to the above target object detection method, the second detection network can detect one or more detection boxes based on an image. If the confidence of one of the detection boxes in the image is greater than a threshold, it is determined that the image is a positive sample (i.e. the second detection network can determine that the target object exists in the image). If the confidence of each detection box in the image is less than or equal to the threshold, it is determined that the image is a negative sample (i.e. the second detection network can determine that the target object does not exist in the image).

[0187] Firstly, the second detection network is used to determine whether the target object exists in N images respectively, and the determination results of the N images are obtained.

[0188] Then, according to the determination results of the N images and whether the target object actually exists in each of the N images, the following data is respectively counted:

[0189] 1. TP: the number of images in which the second detection network determines that the target object exists and the target object actually exists in the N images;

[0190] 2. FP: the number of images in which the second detection network determines that the target object exists and the target object actually does not exist in the N images;

[0191] 3. FN: the number of images in which the second detection network determines that the target object does not exist and the target object actually exists in the N images.

[0192] Finally, the detection accuracy and the recall rate of the second detection network are respectively determined according to the following formulas (5) and (6), wherein:

[0193]

[0194]

[0195] Wherein, P is the detection accuracy of the second detection network; R is the recall rate of the second detection network; TP is the number of images in which the second detection network determines that the target object exists and the target object actually exists in the N images; FP is the number of images in which the second detection network determines that the target object exists and the target object actually does not exist in the N images; FN is the number of images in which the second detection network determines that the target object does not exist and the target object actually exists in the N images.

[0196] Finally, through the detection accuracy and the recall rate of the second detection network, whether the detection accuracy performance of the second detection network meets the requirements can be evaluated. If the detection accuracy performance does not meet the requirements, data can be further supplemented or the network can be modified for training, so as to further improve the detection accuracy of the second detection network on the target object. Similarly, the performance of the first detection network can be evaluated.

[0197] In order to better implement the target object detection method in the embodiments of the present application, on the basis of the target object detection method, a target object detection device is further provided in the embodiments of the present application, as shown in Figure 7 The target object detection device 700 includes:

[0198] The acquisition unit 701 is configured to acquire a state image of a preset area.

[0199] The extraction unit 702 is configured to perform feature extraction processing according to the state image to obtain image features of the state image.

[0200] predicting unit 703 is configured to perform prediction processing according to the image feature to obtain a first confidence degree that the object in the preset region is a preset first object category and a second confidence degree that the object in the preset region is a preset second object category.

[0201] determining unit 704 is configured to determine whether the target object exists in the preset region according to the first confidence degree and the second confidence degree.

[0202] In a possible implementation of the present application, before the step of determining whether the target object exists in the preset region according to the first confidence degree and the second confidence degree, the predicting unit 703 is specifically configured to:

[0203] perform prediction processing according to the image feature to obtain a third confidence degree that the object in the preset region is a preset target object category, wherein the first object category and the second object category are subordinated to the target object category.

[0204] In a possible implementation of the present application, the determining unit 704 is specifically configured to:

[0205] determine whether the target object exists in the preset region according to the first confidence degree, the second confidence degree and the third confidence degree.

[0206] In a possible implementation of the present application, the determining unit 704 is specifically configured to:

[0207] obtain a confidence degree with the largest value from the first confidence degree, the second confidence degree and the third confidence degree as a target confidence degree that the target object exists in the preset region.

[0208] detect whether the target confidence degree is greater than a first preset threshold.

[0209] when it is detected that the target confidence degree is greater than the first preset threshold, determine that the target object exists in the preset region.

[0210] when it is detected that the target confidence degree is less than or equal to the first preset threshold, determine that the target object does not exist in the preset region.

[0211] In a possible implementation of the present application, the predicting unit 703 is specifically configured to:

[0212] invoke a first prediction subnetwork in the trained first detection network to perform prediction processing according to the image feature to obtain the first confidence degree and the second confidence degree.

[0213] The second prediction sub-network in the first detection network after training is called to perform prediction processing according to the image feature, to obtain the third confidence, wherein the first prediction sub-network and the second prediction sub-network are parallel branches.

[0214] In a possible implementation of the present application, the determination unit 704 is specifically configured to:

[0215] perform prediction processing according to the first confidence and the second confidence, to obtain a fourth confidence that the object existing in the preset region is a preset target object category;

[0216] detect whether the fourth confidence is greater than a second preset threshold;

[0217] when it is detected that the fourth confidence is greater than the second preset threshold, determine that the target object exists in the preset region;

[0218] when it is detected that the fourth confidence is less than or equal to the second preset threshold, determine that the target object does not exist in the preset region.

[0219] In a possible implementation of the present application, the prediction unit 703 is specifically configured to:

[0220] call a third prediction sub-network in the second detection network after training to perform prediction processing according to the image feature, to obtain the first confidence and the second confidence;

[0221] call a fourth prediction sub-network in the second detection network after training to perform prediction processing according to the first confidence and the second confidence, to obtain the fourth confidence, wherein the third prediction sub-network and the fourth prediction sub-network are cascaded branches.

[0222] In a possible implementation of the present application, the target object detection apparatus 700 further includes a training unit (not shown in the figure), which is specifically configured to:

[0223] obtain a training data set, wherein the training data set includes a first sample image of the first object category and a second sample image of the second object category;

[0224] obtain a data difference value between the first sample image and the second sample image;

[0225] when the data difference value is greater than a preset quantity threshold, perform augmentation on a sample image of a target sample category to obtain an updated training data set, wherein the target sample category refers to a category with the least number of sample images in the first object category and the second object category;

[0226] Train the first preset network according to the updated training data set, to obtain the second detection network.

[0227] In a possible implementation of the present application, the target object detection apparatus 700 further comprises an alarm unit (not shown in the figure), which is specifically configured to:

[0228] When it is determined that the target object exists in the preset area, output abnormal alarm information.

[0229] In implementation, the above various units can be implemented as independent entities, or can be combined as the same or several entities, and the specific implementation of the above various units can be referred to the method embodiments above, which will not be described here.

[0230] Since the target object detection apparatus can perform the method of the present application as described above, Figures 1 to 6 According to the steps of the target object detection method in any embodiment, the method of the present application as described above can be implemented. Figures 1 to 6 According to the beneficial effects of the target object detection method in any embodiment, the specific description is referred to the above, which will not be described here.

[0231] In addition, in order to better implement the target object detection method in the embodiments of the present application, based on the target object detection method, the embodiments of the present application further provide an electronic device, which is described in detail with reference to Figure 8 , Figure 8 A structural schematic diagram of the electronic device in the embodiments of the present application is shown, and specifically, the electronic device provided by the embodiments of the present application comprises a processor 801, which is configured to implement the steps of the target object detection method as described above when executing the computer program stored in the memory 802. Figures 1 to 6 According to the steps of the target object detection method in any embodiment; or the processor 801 is configured to implement the functions of the units in the embodiments when executing the computer program stored in the memory 802. Figure 7

[0232] For example, the computer program can be divided into one or more modules / units, one or more modules / units are stored in the memory 802 and executed by the processor 801 to complete the embodiments of the present application. One or more modules / units can be a series of computer program instruction segments capable of completing a specific function, which is used to describe the execution process of the computer program in the computer device.

[0233] ​The electronic device can include, but is not limited to, a processor 801, a memory 802. Those skilled in the art can understand that the above-mentioned electronic device is only an example and does not constitute a limitation on the electronic device, and can include more or fewer components than the diagram, or combine certain components, or different components, for example, the electronic device can also include input / output devices, network access devices, buses, etc. The processor 801, the memory 802, the input / output device, and the network access device are connected through the bus.

[0234] The processor 801 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The processor is the control center of the electronic device, and connects all parts of the electronic device through various interfaces and lines.

[0235] The memory 802 can be used to store computer programs and / or modules, and the processor 801 realizes various functions of the computer device by running or executing the computer programs and / or modules stored in the memory 802, and calling the data stored in the memory 802. The memory 802 can mainly include a program storage area and a data storage area, wherein the program storage area can store operating systems, at least one application program required by a function (such as a sound playing function, an image playing function, etc.), etc.; the data storage area can store data created according to the use of the electronic device (such as audio data, video data, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one magnetic disk storage device, a flash memory device, or other volatile solid-state memory device.

[0236] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-mentioned target object detection device, electronic device and corresponding units can be referred to as Figures 1 to 6 The description of the target object detection method corresponding to any embodiment is not repeated here.

[0237] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructions, or by controlling relevant hardware by the instructions, which can be stored in a computer readable storage medium and loaded and executed by a processor.

[0238] To this end, an embodiment of the present application provides a computer readable storage medium, which stores a plurality of instructions capable of being loaded by a processor to execute the method of the present application as Figures 1 to 6 Corresponding to the steps in the target object detection method in any embodiment, the specific operations can be referred to the method of the present application as Figures 1 to 6 Corresponding to the description of the target object detection method in any embodiment, no longer be repeated here.

[0239] The computer readable storage medium can include a read only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0240] Due to the instructions stored in the computer readable storage medium, the method of the present application as Figures 1 to 6 Corresponding to the steps in the target object detection method in any embodiment, the specific operations can be referred to the method of the present application as Figures 1 to 6 Corresponding to the beneficial effects of the target object detection method in any embodiment, see the foregoing description, no longer be repeated here.

[0241] The above describes in detail the target object detection method, device, electronic equipment and computer readable storage medium provided by the embodiments of the present application. The principle and implementation mode of the present application are described by applying specific examples in this paper. The above embodiment is only used to help understand the method and core idea of the present application; at the same time, for those skilled in the art, according to the idea of the present application, the specific implementation mode and application range will be changed; the above description should not be understood as the limitation of the present application.

Claims

1. A target object detection method characterized by, The method comprises: acquiring a state image of a preset region; performing feature extraction processing according to the state image to obtain image features of the state image; performing prediction processing according to the image features to obtain a first confidence degree that an object existing in the preset region is a preset first object category and a second confidence degree that the object is a preset second object category; the first object category and the second object category indicate different placement states of the same object; determining whether a target object exists in the preset region according to the first confidence degree and the second confidence degree.

2. The target object detection method according to claim 1, characterized by, Before the determining whether the target object exists in the preset region according to the first confidence degree and the second confidence degree, the method further comprises: performing prediction processing according to the image features to obtain a third confidence degree that the object existing in the preset region is a preset target object category, wherein the first object category and the second object category are subordinated to the target object category; the determining whether the target object exists in the preset region according to the first confidence degree and the second confidence degree comprises: determining whether the target object exists in the preset region according to the first confidence degree, the second confidence degree and the third confidence degree.

3. The target object detection method according to claim 2, characterized by, The determining whether the target object exists in the preset region according to the first confidence degree, the second confidence degree and the third confidence degree comprises: acquiring a confidence degree with the largest value from the first confidence degree, the second confidence degree and the third confidence degree as a target confidence degree that the target object exists in the preset region; detecting whether the target confidence degree is greater than a first preset threshold; when it is detected that the target confidence degree is greater than the first preset threshold, determining that the target object exists in the preset region; when it is detected that the target confidence degree is less than or equal to the first preset threshold, determining that the target object does not exist in the preset region.

4. The target object detection method according to claim 2, characterized by, The performing prediction processing according to the image features to obtain the first confidence degree that the object existing in the preset region is the preset first object category and the second confidence degree that the object is the preset second object category comprises: calling a first prediction subnetwork in a trained first detection network to perform prediction processing according to the image features to obtain the first confidence degree and the second confidence degree; The performing prediction processing according to the image features to obtain the third confidence degree that the object existing in the preset region is the preset target object category comprises: calling a second prediction subnetwork in the trained first detection network to perform prediction processing according to the image features to obtain the third confidence degree, wherein the first prediction subnetwork and the second prediction subnetwork are parallel branches.

5. The target object detection method according to claim 1, wherein The determining whether the target object exists in the preset region according to the first confidence degree and the second confidence degree comprises: performing prediction processing according to the first confidence degree and the second confidence degree to obtain a fourth confidence degree that the object existing in the preset region is the preset target object category; detecting whether the fourth confidence degree is greater than a second preset threshold; when it is detected that the fourth confidence degree is greater than the second preset threshold, determining that the target object exists in the preset region; When it is detected that the fourth confidence is less than or equal to a second preset threshold, it is determined that the preset region does not exist a target object.

6. The target object detection method according to claim 5, wherein The prediction processing according to the image features obtains a first confidence that the object existing in the preset region is a preset first object category and a second confidence that the object existing in the preset region is a preset second object category. The third prediction sub-network in the trained second detection network is called to perform prediction processing according to the image features to obtain the first confidence and the second confidence. The fourth confidence that the object existing in the preset region is a preset target object category is obtained through prediction processing according to the first confidence and the second confidence. The fourth prediction sub-network in the trained second detection network is called to perform prediction processing according to the first confidence and the second confidence to obtain the fourth confidence, wherein the third prediction sub-network and the fourth prediction sub-network are cascade branches.

7. The target object detection method according to claim 6, wherein The method further comprises: obtaining a training data set, wherein the training data set comprises first sample images of the first object category and second sample images of the second object category; obtaining a data difference between the first sample images and the second sample images; when the data difference is greater than a preset quantity threshold, performing augmentation on sample images of a target sample category to obtain an updated training data set, wherein the target sample category refers to a category with the least number of sample images among the first object category and the second object category; training the first preset network according to the updated training data set to obtain the second detection network.

8. The target object detection method according to any one of claims 1 to 7, characterized by, The method further comprises: when it is determined that the preset region exists a target object, outputting an abnormal alarm information.

9. An object detection device, characterized by comprising: The target object detection device comprises: an acquisition unit configured to acquire a state image of a preset region; an extraction unit configured to perform feature extraction processing on the state image to obtain image features of the state image; a prediction unit configured to perform prediction processing on the image features to obtain a first confidence that an object existing in the preset region is a preset first object category and a second confidence that the object existing in the preset region is a preset second object category, wherein the first object category and the second object category indicate different placement states of the same object; a determination unit configured to determine whether the preset region exists a target object according to the first confidence and the second confidence.

10. An electronic device, comprising: a processor and a memory, wherein the memory stores a computer program, and the processor calls the computer program in the memory to execute the target object detection method according to any one of claims 1 to 8.

11. A computer readable storage medium, characterized in that, a computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in the target object detection method according to any one of claims 1 to 8.

Images