Target detection method, device and electronic system

Abstract

The present invention provides a target detection method, device and electronic system; wherein the method comprises: extracting image features of an image to be detected; inputting the image features into a preset target detection network, and outputting target classification results, target positioning results and foreground recognition results of the image to be detected; wherein the foreground recognition results include the probability that each position in the image to be detected belongs to the foreground; adjusting the target classification results based on the foreground recognition results; and determining the target detection results of the image to be detected based on the target positioning results and the adjusted target classification results. In this method, the target classification results are adjusted based on the foreground recognition results of the image to be detected, so that the features belonging to the foreground area can be strengthened, the features belonging to the background area can be weakened, the probability of misidentifying the background as a target is reduced, and the target detection effect is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a target detection method, device and electronic system. Background Art

[0002] The target detection task mainly includes two subtasks: target classification and target detection box regression. Common target detection tasks mainly include face detection tasks, pedestrian detection tasks, etc.; since the detection targets of these detection tasks are relatively single, the detection effect is better; while for target detection of general objects, due to the wide variety of targets, the detection performance of the network is affected, and it is easy to have problems such as missed detection or false detection of targets, resulting in poor detection effect. Summary of the invention

[0003] In view of this, an object of the present invention is to provide a target detection method, device and electronic system to improve the target detection effect.

[0004] In a first aspect, an embodiment of the present invention provides a target detection method, the method comprising: extracting image features of an image to be detected; inputting the image features into a preset target detection network, and outputting target classification results, target positioning results, and foreground recognition results of the image to be detected; wherein the foreground recognition results include the probability that each position in the image to be detected belongs to the foreground; adjusting the target classification results based on the foreground recognition results; and determining the target detection results of the image to be detected based on the target positioning results and the adjusted target classification results.

[0005] The above-mentioned target detection network includes a classification subnetwork and a position regression subnetwork; the above-mentioned step of inputting image features into a preset target detection network and outputting target classification results, target positioning results and foreground recognition results of the image to be detected includes: inputting image features into the classification subnetwork and outputting target classification results of the image to be detected; inputting image features into the position regression subnetwork and outputting target positioning results and foreground recognition results of the image to be detected.

[0006] The above-mentioned position regression subnetwork includes: a first convolution module, and a first convolution layer and a second convolution layer respectively connected to the first convolution module; wherein the first convolution module includes a plurality of convolution layers connected in series; the above-mentioned step of inputting the image features into the position regression subnetwork and outputting the target positioning result and the foreground recognition result of the image to be detected includes: inputting the image features into the first convolution module and outputting the first intermediate result; inputting the first intermediate result into the first convolution layer and outputting the target positioning result of the image to be detected; inputting the first intermediate result into the second convolution layer and outputting the foreground recognition result of the image to be detected.

[0007] The above-mentioned classification subnetwork includes: a second convolution module, and a third convolution layer connected to the second convolution module; wherein the second convolution module includes multiple convolution layers connected in series; the above-mentioned step of inputting image features into the classification subnetwork and outputting the target classification result of the image to be detected includes: inputting image features into the second convolution module and outputting a second intermediate result; inputting the second intermediate result into the third convolution layer and outputting the target classification result of the image to be detected.

[0008] The foreground recognition result includes: the parameter value of the probability that each pixel in the image to be detected belongs to the foreground; the target classification result includes: the parameter value of the probability that each pixel in the image to be detected belongs to each preset target category; the step of adjusting the target classification result based on the foreground recognition result includes: for each pixel, multiplying the parameter value of the probability that the pixel belongs to each preset target category by the parameter value of the probability that the pixel belongs to the foreground, to obtain the adjusted target classification result.

[0009] Before the step of obtaining the adjusted target classification result by multiplying the parameter value of the probability that each pixel point belongs to each preset target category by the parameter value of the probability that the pixel point belongs to the foreground for each pixel point, the method further includes: inputting the foreground recognition result and the target classification result into preset activation functions respectively, so as to map each parameter value in the foreground recognition result and the target classification result into a specified numerical range through the activation function, so as to obtain the mapped foreground recognition result and the target classification result; wherein the maximum value of the numerical range is one and the minimum value is zero.

[0010] The above-mentioned step of extracting image features of the image to be detected includes: inputting the image to be detected into a preset residual neural network to obtain an initial result; inputting the initial result into a preset feature pyramid network to output a multi-scale feature map; and splicing the feature map according to the scale of the multi-scale feature map to obtain the image features of the image to be detected.

[0011] In a second aspect, an embodiment of the present invention provides a target detection device, which includes: a feature extraction module, used to extract image features of an image to be detected; a result output module, used to input the image features into a preset target detection network, and output a target classification result, a target positioning result, and a foreground recognition result of the image to be detected; wherein the foreground recognition result includes the probability that each position in the image to be detected belongs to the foreground; a result adjustment module, used to adjust the target classification result based on the foreground recognition result; and determine the target detection result of the image to be detected based on the target positioning result and the adjusted target classification result.

[0012] In a third aspect, an embodiment of the present invention provides an electronic system, the electronic system comprising: a processing device and a storage device; a computer program is stored on the storage device, and the computer program executes the above-mentioned target detection method when the processed device is running.

[0013] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is run by a processing device, the steps of the above-mentioned target detection method are executed.

[0014] The embodiments of the present invention bring the following beneficial effects:

[0015] The above-mentioned target detection method, device and electronic system, after the image features of the image to be detected are input into the preset target detection network, the target classification result, target positioning result and foreground recognition result of the image to be detected are output; wherein the foreground recognition result includes the probability that each position in the image to be detected belongs to the foreground; then the target classification result is adjusted based on the foreground recognition result; based on the target positioning result and the adjusted target classification result, the target detection result of the image to be detected is determined. In this method, the target classification result is adjusted based on the foreground recognition result of the image to be detected, so that the features belonging to the foreground area can be strengthened, the features belonging to the background area can be weakened, the probability of misidentifying the background as the target is reduced, and the target detection effect is improved.

[0016] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.

[0017] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 A schematic diagram of the structure of an electronic system provided by an embodiment of the present invention;

[0020] Figure 2 A flow chart of a target detection method provided by an embodiment of the present invention;

[0021] Figure 3 A schematic diagram of the structure of a target detection network provided by an embodiment of the present invention;

[0022] Figure 4 A schematic diagram of the structure of a target detection device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0024] General object detection is an important and far-reaching task, which can be seen as a more general and extensive version of various detection tasks. If the general object detection task can be solved well, it will help various specific tasks, such as face detection and pedestrian detection. For general object detection, due to the wide variety of targets, the detection performance of the network is affected, and it is easy to miss or misdetect targets, resulting in poor detection results. The target detection task mainly includes two tasks: classification and bounding box regression. Therefore, in order to improve the detection effect of general object detection, we can start by improving the quality of the classification task or the quality of the bounding box regression task.

[0025] Based on this, an embodiment of the present invention provides a target detection method, device and electronic system; this technology can be applied to target detection tasks in various scenarios, such as face detection, pedestrian detection or detection of other targets.

[0026] Embodiment 1:

[0027] First, refer to Figure 1 An exemplary electronic system 100 for implementing the target detection method, apparatus, and electronic system according to the embodiments of the present invention is described.

[0028] like Figure 1 The electronic system 100 includes one or more processing devices 102, one or more storage devices 104, an input device 106, an output device 108, and may also include one or more image acquisition devices 110. These components are interconnected through a bus system 112 and / or other forms of connection mechanisms (not shown). It should be noted that Figure 1The components and structure of the electronic system 100 shown are merely exemplary and not limiting. The electronic system may also have other components and structures as required.

[0029] The processing device 102 can be a gateway, or a smart terminal, or a device including a central processing unit (CPU) or other forms of processing units with data processing capabilities and / or instruction execution capabilities. It can process data of other components in the electronic system 100 and control other components in the electronic system 100 to perform desired functions.

[0030] The storage device 104 may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processing device 102 may run the program instructions to implement the client functions and / or other desired functions in the embodiments of the present invention (implemented by the processing device) described below. Various applications and various data, such as various data used and / or generated by the application, may also be stored in the computer-readable storage medium.

[0031] The input device 106 may be a device used by a user to input instructions, and may include one or more of a keyboard, a mouse, a microphone, a touch screen, and the like.

[0032] The output device 108 may output various information (eg, images or sounds) to the outside (eg, a user), and may include one or more of a display, a speaker, and the like.

[0033] The image acquisition device 110 can acquire preview video frames or picture data (such as images to be detected or target video frames), and store the acquired preview video frames or image data in the storage device 104 for use by other components.

[0034] Exemplarily, the components in the example electronic system for implementing the target detection method, device and electronic system according to the embodiment of the present invention can be integrated or dispersed, such as integrating the processing device 102, the storage device 104, the input device 106 and the output device 108 into one, and setting the image acquisition device 110 at a specified position where the image can be acquired. When the components in the above electronic system are integrated, the electronic system can be implemented as a smart terminal such as a camera, a smart phone, a tablet computer, a computer, a vehicle-mounted terminal, a camera, etc.

[0035] Embodiment 2:

[0036] See also Figure 2 A flow chart of a target detection method is shown, the method comprising the following steps:

[0037] Step S202, extracting image features of the image to be detected;

[0038] The image to be detected can be input into a pre-trained feature extraction network to output image features; before the image to be detected is input into the feature extraction network, it may also need to be preprocessed, for example, scaling the image to be detected to a specified resolution, for example, the resolution can be 800*1333. During the training of the feature extraction network, the number of sample images can also be expanded through specific operations, for example, the existing sample images can be flipped with a certain probability. In specific implementation, the sample images can be flipped horizontally with a probability of 50%. The above-mentioned feature extraction network can be implemented through a variety of networks. In this embodiment, it can be implemented through a RetinaNet network, a resnet-50+FPN network structure, etc.

[0039] Step S204, inputting the image features into a preset target detection network, and outputting the target classification result, target positioning result and foreground recognition result of the image to be detected; wherein the foreground recognition result includes the probability that each position in the image to be detected belongs to the foreground;

[0040] The target classification result may be in the form of a multidimensional matrix. Specifically, the target classification result may include a classification result for each pixel in the image to be detected; the classification result for each pixel includes a score or probability that the pixel belongs to each preset category. The target positioning result may include information such as the position and size of multiple positioning boxes.

[0041] In addition to outputting target classification results and target positioning results, the above target detection network also outputs foreground recognition results. Foreground recognition can be understood as a binary classification problem, where each pixel of the image to be detected is divided into foreground and background; each pixel corresponds to a numerical value, which is used to indicate the probability that each position in the image to be detected belongs to the foreground. For example, in actual implementation, the larger the numerical value corresponding to the pixel, the greater the probability that the pixel belongs to the foreground; the smaller the numerical value corresponding to the pixel, the greater the probability that the pixel belongs to the background.

[0042] Step S206, adjusting the target classification result based on the foreground recognition result; and determining the target detection result of the image to be detected based on the target positioning result and the adjusted target classification result.

[0043] In most cases, the target contained in the image belongs to the foreground; based on this, the foreground recognition result can strengthen the features of the foreground area in the target classification result, and then weaken the features in the background area; for example, the features belonging to the foreground area can be multiplied by a larger value, and the features belonging to the background area can be multiplied by a smaller value. In this way, the area where the background is mistakenly identified as the target can be weakened, thereby avoiding the situation where the background is mistakenly identified as the target and reducing the probability of misidentification.

[0044] In the above target detection method, after the image features of the image to be detected are input into the preset target detection network, the target classification result, target positioning result and foreground recognition result of the image to be detected are output; wherein the foreground recognition result includes the probability that each position in the image to be detected belongs to the foreground; then the target classification result is adjusted based on the foreground recognition result; based on the target positioning result and the adjusted target classification result, the target detection result of the image to be detected is determined. In this method, the target classification result is adjusted based on the foreground recognition result of the image to be detected, so that the image features belonging to the foreground area can be strengthened, the image features belonging to the background area can be weakened, the probability of misidentifying the background as the target is reduced, and the target detection effect is improved.

[0045] Embodiment three:

[0046] This embodiment focuses on describing the network structure of the target detection network and the process of outputting results based on the network results.

[0047] First, the target detection network includes a classification subnetwork and a position regression subnetwork; the image features are input into the classification subnetwork to output the target classification result of the image to be detected; the image features are input into the position regression subnetwork to output the target positioning result and foreground recognition result of the image to be detected. The difference from the target detection network in the related art is that the position regression subnetwork needs to output the target positioning result and foreground recognition result; for this purpose, the position regression subnetwork needs to set up two branch structures; see Figure 3 As shown, the position regression subnetwork includes: a first convolution module, and a first convolution layer and a second convolution layer respectively connected to the first convolution module; wherein the first convolution module includes multiple convolution layers connected in series; for example, the first convolution module may specifically include four 3*3 two-dimensional convolution layers; the first convolution layer may specifically be a 1*1 two-dimensional convolution layer; the second convolution layer may also be a 1*1 two-dimensional convolution layer.

[0048] Based on the above-mentioned position regression subnetwork, the image features can be input into the first convolution module to output the first intermediate result; the first intermediate result is input into the first convolution layer to output the target positioning result of the image to be detected; the first intermediate result is input into the second convolution layer to output the foreground recognition result of the image to be detected. The target positioning result may include the score of each pixel point in the image to be detected belonging to each preset target category, and the higher the score, the greater the probability that the pixel point belongs to the target category. Similarly, the above-mentioned foreground recognition result may include the score of each pixel point in the image to be detected belonging to the foreground or background, which can be set to, the higher the score, the greater the probability that the pixel point belongs to the foreground; the lower the score, the greater the probability that the pixel point belongs to the background.

[0049] Continue to refer Figure 3 , the classification subnetwork includes: a second convolution module, and a third convolution layer connected to the second convolution module; wherein the second convolution module includes a plurality of convolution layers connected in series; the second convolution module may specifically include four 3*3 two-dimensional convolution layers; the third convolution layer may specifically be a 1*1 two-dimensional convolution layer. Based on the classification subnetwork, the image features may be input into the second convolution module, and the second intermediate result may be output; the second intermediate result may be input into the third convolution layer, and the target classification result of the image to be detected may be output.

[0050] In actual implementation, the foreground recognition results output by the position regression subnetwork and the target classification results output by the classification subnetwork are often not probabilities between 0 and 1; however, these values ​​are linearly related to the probabilities, that is, the larger the value, the greater the probability value represented. In order to adjust the target classification results by the foreground recognition results, it is necessary to map the values ​​in the foreground recognition results and the target classification results to a preset value range in advance.

[0051] In a specific implementation, the foreground recognition result includes: the parameter value of the probability that each pixel in the image to be detected belongs to the foreground; the target classification result includes: the parameter value of the probability that each pixel in the image to be detected belongs to each preset target category. The parameter value here can also be understood as an indication value of the probability, and the numerical range of the parameter value is often very large, for example, it can be 0-1000, and there can also be negative numbers. At this time, the parameter value can be linearly mapped to the range of 0-1 through an activation function, such as a sigmoid function. Specifically, the foreground recognition result and the target classification result are respectively input into the preset activation function, so that each parameter value in the foreground recognition result and the target classification result is mapped to the specified numerical range through the activation function, and the mapped foreground recognition result and the target classification result are obtained; wherein the maximum value of the numerical range is one and the minimum value is zero.

[0052] Taking the sigmoid function as an example, firstly, the foreground recognition result is input into the sigmoid function, and each parameter value in the foreground recognition result is mapped through the sigmoid function, and the foreground recognition result after mapping is output; then, the target classification result is input into the sigmoid function, and each parameter value in the target classification result is mapped through the sigmoid function, and the target classification result after mapping is output.

[0053] The target classification result after mapping processing is adjusted by the foreground recognition result after mapping processing; specifically, for each pixel point, the parameter value of the probability that the pixel point belongs to each preset target category is multiplied by the parameter value of the probability that the pixel point belongs to the foreground, and the adjusted target classification result is obtained. If there are 80 preset target categories in total, one pixel point corresponds to 80 parameter values, and these 80 parameter values ​​are multiplied by the parameter value of the probability that the pixel point belongs to the foreground. If the parameter value of the probability that the pixel point belongs to the foreground is large, it means that the probability that the pixel point belongs to the foreground is high. After the parameter of the probability that the pixel point belongs to each target category is multiplied by the parameter value of the probability that the pixel point belongs to the foreground, the parameter of the probability that the pixel point belongs to each target category will be enhanced.

[0054] However, if the parameter value of the probability that the pixel belongs to the foreground is small or even close to zero, it means that the probability that the pixel belongs to the foreground is small and the pixel belongs to the background to a large extent. Since most targets belong to the foreground, the pixel should not be identified as a target. After multiplying the parameter of the probability that the pixel belongs to each target category by the parameter value of the probability that the pixel belongs to the foreground, the parameter of the probability that the pixel belongs to each target category will be weakened, and the probability that the pixel is identified as a target will be reduced, thereby avoiding misidentification of the pixel as a target.

[0055] During the training process of the above target detection network, the loss value can be calculated in the same loss calculation method as RetinaNet. Specifically, the loss function used can be called Focal loss; the Focal loss function is an improvement on the cross entropy function, and an adjustment item is added before the cross entropy function to adjust the contribution of different probability templates to the loss.

[0056] In the network structure of the target detection network provided in this embodiment, a new network branch is added in the position regression subnetwork to output the foreground recognition result of the image to be detected. The network score can also be called the objectness branch. This branch can reduce the false positive rate as a whole, improve the recall rate of the model, and reduce the miss rate; the above method can significantly improve the accuracy of target detection classification, reduce the false detection and miss rates, and thus improve the overall target detection effect.

[0057] Embodiment 4:

[0058] This embodiment focuses on the process of extracting image features. The extraction of image features is mainly achieved through the backbone network. In this embodiment, the image features can be extracted by using the RetinaNet network, or by using a residual neural network and a feature pyramid network (FPN) similar to the RetinaNet network. Figure 3 The backbone network shown in the figure can be implemented by a residual neural network or other networks. First, the image to be detected is input into a preset residual neural network to obtain an initial result; then the initial result is input into a preset feature pyramid network to output a multi-scale feature map; according to the scale of the multi-scale feature map, the feature map is spliced ​​to obtain the image features of the image to be detected. The residual neural network here can be a Resnet-50 network or other residual neural networks.

[0059] The multi-scale feature map includes multiple feature maps, which may also be referred to as feature maps. Each feature map has a different scale. The multiple feature maps may be concatenated in sequence according to the order of the feature map scales from large to small, and then input into the target detection network in the above embodiment.

[0060] Embodiment five:

[0061] Corresponding to the above method embodiment, see Figure 4 A schematic diagram of the structure of a target detection device is shown, the device comprising:

[0062] A feature extraction module 40 is used to extract image features of the image to be detected;

[0063] The result output module 42 is used to input the image features into a preset target detection network, and output the target classification result, target positioning result and foreground recognition result of the image to be detected; wherein the foreground recognition result includes the probability that each position in the image to be detected belongs to the foreground;

[0064] The result adjustment module 44 is used to adjust the target classification result based on the foreground recognition result; and determine the target detection result of the image to be detected based on the target positioning result and the adjusted target classification result.

[0065] In the above target detection device, after the image features of the image to be detected are input into the preset target detection network, the target classification result, target positioning result and foreground recognition result of the image to be detected are output; wherein the foreground recognition result includes the probability that each position in the image to be detected belongs to the foreground; then the target classification result is adjusted based on the foreground recognition result; based on the target positioning result and the adjusted target classification result, the target detection result of the image to be detected is determined. In this method, the target classification result is adjusted based on the foreground recognition result of the image to be detected, so that the features belonging to the foreground area can be strengthened, the features belonging to the background area can be weakened, the probability of misidentifying the background as the target is reduced, and the target detection effect is improved.

[0066] The above-mentioned target detection network includes a classification subnetwork and a position regression subnetwork; the above-mentioned result output module is also used to: input image features into the classification subnetwork, and output the target classification result of the image to be detected; input image features into the position regression subnetwork, and output the target positioning result and foreground recognition result of the image to be detected.

[0067] The above-mentioned position regression subnetwork includes: a first convolution module, and a first convolution layer and a second convolution layer respectively connected to the first convolution module; wherein the first convolution module includes multiple convolution layers connected in series; the above-mentioned result output module is also used to: input the image features into the first convolution module and output the first intermediate result; input the first intermediate result into the first convolution layer and output the target positioning result of the image to be detected; input the first intermediate result into the second convolution layer and output the foreground recognition result of the image to be detected.

[0068] The above-mentioned classification subnetwork includes: a second convolution module, and a third convolution layer connected to the second convolution module; wherein the second convolution module includes multiple convolution layers connected in series; the above-mentioned result output module is also used to: input image features into the second convolution module and output a second intermediate result; input the second intermediate result into the third convolution layer and output a target classification result of the image to be detected.

[0069] The foreground recognition result includes: the parameter value of the probability that each pixel in the image to be detected belongs to the foreground; the target classification result includes: the parameter value of the probability that each pixel in the image to be detected belongs to each preset target category; the result adjustment module is also used for: for each pixel, multiplying the parameter value of the probability that the pixel belongs to each preset target category by the parameter value of the probability that the pixel belongs to the foreground, to obtain the adjusted target classification result.

[0070] The above-mentioned device also includes a mapping module, which is used to: input the foreground recognition result and the target classification result into a preset activation function respectively, so as to map each parameter value in the foreground recognition result and the target classification result into a specified numerical range through the activation function, and obtain the mapped foreground recognition result and the target classification result; wherein the maximum value of the numerical range is one and the minimum value is zero.

[0071] The feature extraction module is also used to: input the image to be detected into a preset residual neural network to obtain an initial result; input the initial result into a preset feature pyramid network to output a multi-scale feature map; and splice the feature map according to the scale of the multi-scale feature map to obtain the image features of the image to be detected.

[0072] The image processing device provided by the embodiment of the present invention has the same technical features as the image processing method provided by the above embodiment, so it can also solve the same technical problems and achieve the same technical effects.

[0073] Embodiment six:

[0074] An embodiment of the present invention provides an electronic system, which includes: a processing device and a storage device; the storage device stores a computer program, and the computer program executes the above-mentioned target detection method when the processed device runs.

[0075] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the electronic system described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0076] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processing device, the steps of the target detection method described above are executed.

[0077] The computer program products of the target detection methods, devices, and electronic systems provided in the embodiments of the present invention include a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods described in the previous method embodiments. The specific implementation can be found in the method embodiments, which will not be repeated here.

[0078] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0079] In addition, in the description of the embodiments of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, and it can be the internal communication of two components. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0080] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0081] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0082] Finally, it should be noted that the above embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above embodiments, those skilled in the art should understand that any person skilled in the art can still modify the technical solutions recorded in the above embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A target detection method, characterized in that: The method comprises: Extracting image features of the image to be detected; The image features are input into a preset target detection network, and the target classification result, target positioning result and foreground recognition result of the image to be detected are output; wherein the foreground recognition result includes: a parameter value of the probability that each pixel point in the image to be detected belongs to the foreground; the target classification result includes: a parameter value of the probability that each pixel point in the image to be detected belongs to each preset target category; For each pixel point, the parameter value of the probability that the pixel point belongs to each preset target category is multiplied by the parameter value of the probability that the pixel point belongs to the foreground to obtain the adjusted target classification result; based on the target positioning result and the adjusted target classification result, the target detection result of the image to be detected is determined.

2. The method according to claim 1, characterized in that: The target detection network includes a classification subnetwork and a position regression subnetwork; The step of inputting the image features into a preset target detection network and outputting the target classification result, target positioning result and foreground recognition result of the image to be detected comprises: The image features are input into the classification subnetwork, and the target classification result of the image to be detected is output; the image features are input into the position regression subnetwork, and the target positioning result and foreground recognition result of the image to be detected are output.

3. The method according to claim 2, characterized in that The position regression subnetwork includes: a first convolution module, and a first convolution layer and a second convolution layer respectively connected to the first convolution module; wherein the first convolution module includes a plurality of convolution layers connected in series; The step of inputting the image features into the position regression subnetwork and outputting the target positioning result and foreground recognition result of the image to be detected includes: Inputting the image features into the first convolution module and outputting a first intermediate result; The first intermediate result is input into the first convolution layer, and the target positioning result of the image to be detected is output; the first intermediate result is input into the second convolution layer, and the foreground recognition result of the image to be detected is output.

4. The method according to claim 2, characterized in that: The classification subnetwork includes: a second convolution module, and a third convolution layer connected to the second convolution module; wherein the second convolution module includes a plurality of convolution layers connected in series; The step of inputting the image features into the classification subnetwork and outputting the target classification result of the image to be detected includes: Inputting the image features into the second convolution module, and outputting a second intermediate result; The second intermediate result is input into the third convolutional layer, and the target classification result of the image to be detected is output.

5. The method according to claim 1, characterized in that Before the step of obtaining the adjusted target classification result by multiplying, for each pixel, the parameter value of the probability that the pixel belongs to each preset target category by the parameter value of the probability that the pixel belongs to the foreground, the method further includes: The foreground recognition result and the target classification result are respectively input into a preset activation function, so that each parameter value in the foreground recognition result and the target classification result is mapped into a specified numerical range through the activation function to obtain the mapped foreground recognition result and the target classification result; wherein the maximum value of the numerical range is one and the minimum value is zero.

6. The method according to claim 1, characterized in that The step of extracting image features of the image to be detected includes: Inputting the image to be detected into a preset residual neural network to obtain an initial result; Inputting the initial result into a preset feature pyramid network and outputting a multi-scale feature map; According to the scales of the multi-scale feature maps, the feature maps are spliced ​​to obtain image features of the image to be detected.

7. A target detection device, characterized in that: The device comprises: A feature extraction module, used to extract image features of the image to be detected; A result output module is used to input the image features into a preset target detection network, and output the target classification result, target positioning result and foreground recognition result of the image to be detected; wherein the foreground recognition result includes: a parameter value of the probability that each pixel point in the image to be detected belongs to the foreground; the target classification result includes: a parameter value of the probability that each pixel point in the image to be detected belongs to each preset target category; The result adjustment module is used to multiply the parameter value of the probability that each pixel point belongs to each preset target category by the parameter value of the probability that the pixel point belongs to the foreground, so as to obtain the adjusted target classification result; based on the target positioning result and the adjusted target classification result, determine the target detection result of the image to be detected.

8. An electronic system, characterized in that: The electronic system includes: a processing device and a storage device; The storage device stores a computer program, and when the computer program is executed by the processing device, the target detection method according to any one of claims 1 to 6 is executed.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processing device, the steps of the target detection method according to any one of claims 1 to 6 are executed.

Images