Target detection method, storage medium and electronic equipment

Abstract

The invention relates to the field of artificial intelligence, and particularly discloses a target detection method, a storage medium and electronic equipment. The method comprises the steps that image information and a target detection data set of a to-be-detected target are acquired, the image information comprises a first image, the target detection data set comprises a second image, and a target object type included in the first image is the same as a target object type included in the second image; determining first weight information based on the target detection data set, wherein the first weight information at least comprises true values of space attention weights corresponding to different areas of the second image; a target detection result is determined based on the first weight information and the image information, and the target detection result comprises at least one of a bounding box prediction result, an object category classification result and a confidence coefficient result. Through the target detection method and device, the technical problem that false detection and missing detection are prone to occurring in a target detection algorithm under the complex background condition in the related technology is solved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and more specifically, to a target detection method, a storage medium, and an electronic device. Background Art

[0002] As research on artificial intelligence applications continues to develop, methods such as deep learning have also emerged and are increasingly being used in target detection, natural language processing, speech recognition, and other fields. Deep learning is an indispensable element in the face recognition function of bank ATMs, large-value machines, and other terminals. In these application scenarios, the accuracy and speed of target detection are extremely high.

[0003] Nowadays, deep learning-based target detection algorithms can be roughly divided into two categories: one is a two-stage target detection algorithm based on region extraction, and the other is a one-stage target detection algorithm that directly performs position regression. The former is also called a target detection algorithm based on region design. This type of algorithm converts the target detection problem into a classification problem of local images in the generated proposed region through explicit region proposals, such as the R-CNN algorithm. This algorithm has greatly improved the average accuracy of target detection, but the calculation speed is slow, and the R-CNN algorithm cannot be trained end-to-end. The latter is also called a regression-based target detection algorithm. Unlike the two-stage target detection algorithm that directly generates the proposed region, this type of algorithm regards the target detection task as a regression task for the entire image, such as the SSD algorithm and the YOLO series of algorithms. The SSD algorithm is more accurate, but its debugging is extremely dependent on experience. Secondly, it is not sensitive to small targets, and insufficient feature extraction will lead to a decrease in its accuracy. The YOLOv5 algorithm has a fast detection speed and can achieve real-time monitoring. It also has a good detection effect on small targets and high detection accuracy, which can meet the needs of real-time target detection.

[0004] Since its launch, the YOLOv5 algorithm has achieved good results in the academic field and various industries with its advantages of small model size, low deployment cost, high flexibility, and fast detection speed. However, for images with complex background information, the existing YOLOv5 algorithm does not highlight the feature information of the detected target compared to the background information, and often misses or misdetects during detection, which seriously affects the target detection effect.

[0005] With regard to the technical problem in related technologies that target detection algorithms are prone to false detection and missed detection under complex background conditions, no effective solution has been proposed so far. Summary of the invention

[0006] The main purpose of the present application is to provide a target detection method, a storage medium and an electronic device to solve the technical problem in the related art that the target detection algorithm is prone to false detection and missed detection under complex background conditions.

[0007] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a target detection method is provided, which includes: obtaining image information of a target to be detected and a target detection data set, wherein the image information includes a first image, the target detection data set includes a second image, and the target object type included in the first image is the same as the target object type included in the second image; determining first weight information based on the target detection data set, the first weight information at least including the true value of the spatial attention weight corresponding to different regions of the second image; determining a target detection result based on the first weight information and the image information, wherein the target detection result includes at least one of the following: a bounding box prediction result, an object category classification result, and a confidence result.

[0008] Furthermore, the target detection result is determined based on the first weight information and the image information, including: inputting the image information and the first weight information into a part of the backbone network in the target detection model for feature extraction to obtain output feature information, wherein the backbone network also includes a feature pyramid branch network; inputting the output feature information into the feature pyramid branch network to generate second weight information, the second weight information at least including predicted values ​​of spatial attention weights of different regions of the first image; and inputting the second weight information into the remaining part of the backbone network to obtain the target detection result.

[0009] Furthermore, determining the first weight information based on the target detection data set includes: determining the polyline label information based on the target detection data set, the polyline label information including the brief central structure morphology of the target object in the target detection data set; mapping the polyline labels contained in the polyline label information into continuous points to obtain continuous point information; and performing Gaussian blur on the continuous point information to obtain the first weight information.

[0010] Furthermore, determining the polyline label information based on the target detection data set includes: annotating at least one target object in the target detection data set with a polyline to obtain the polyline label information.

[0011] Furthermore, before inputting the image information and the first weight information into a partial backbone network in the target detection model for feature extraction, the method also includes: obtaining a true value heat map, wherein the true value heat map is determined by the first weight information; performing channel splicing on the true value heat map and the first image to obtain a spliced ​​image; performing data enhancement processing on the spliced ​​image to obtain an enhanced image; splitting the enhanced image into an enhanced true value heat map and an enhanced first image after channel splitting; determining the enhanced true value heat map as the first weight information; and determining the enhanced first image as image information.

[0012] Furthermore, the second weight information is input into the remaining backbone network to obtain the target detection result, including: inputting the second weight information into the remaining backbone network for forward propagation to obtain a feature map; inputting the feature map into the head network of the target detection model to obtain the target detection result.

[0013] Furthermore, after the second weight information is input into the remaining part of the backbone network, the method also includes: constructing a loss function based on the first weight information and the second weight information; determining the loss parameters in this target detection process based on the loss function; and adjusting the network model parameters of the feature pyramid branch network based on the loss parameter feedback.

[0014] In order to achieve the above-mentioned purpose, according to another aspect of the present application, a computer-readable storage medium is provided, the computer-readable storage medium including a stored executable program, wherein when the executable program is running, the device where the computer-readable storage medium is located is controlled to execute the steps of any one of the above-mentioned methods.

[0015] According to another aspect of the present application, an electronic device is provided, comprising: a memory storing an executable program; and a processor for running the program, wherein any one of the above methods is executed when the program is run.

[0016] According to another aspect of the present application, a computer program product is provided, comprising computer instructions, which implement the steps of any one of the above methods when executed by a processor.

[0017] In an embodiment of the present application, a method of weighting spatial attention is adopted, and the true value of the spatial attention weight corresponding to different areas is determined based on the target detection data set, so as to significantly improve the accuracy of target detection, thereby achieving the purpose of being able to focus on the target object more accurately when detecting the first image, thereby achieving the technical effect of being able to more effectively identify and locate the target object when detecting images to be detected with complex and changeable backgrounds, thereby improving the overall detection performance, and further solving the technical problem that the target detection algorithm is prone to false detection and missed detection under complex background conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0019] Figure 1 A hardware structure block diagram of a computer terminal for implementing a target detection method is shown;

[0020] Figure 2 is a flow chart of a target detection method provided according to an embodiment of the present application;

[0021] Figure 3 is a flow chart of a target detection method provided according to an embodiment of the present application;

[0022] Figure 4 is a network structure diagram of a target detection method provided in an embodiment of the present application;

[0023] Figure 5 is a network structure diagram of a target detection method provided in an embodiment of the present application;

[0024] Figure 6 is a structural block diagram of an electronic device according to an embodiment of the present application;

[0025] Figure 7 It is a structural diagram of a target detection device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0028] First, some nouns or terms that appear in the description of the embodiments of the present application are subject to the following explanations:

[0029] YOLO: The full name is "You Only Look Once", which is a classic object detection model. Its core idea is to regard object detection as a regression problem and directly perform predictions on the image with a fully connected network without generating candidate regions first. YOLO divides the input image into multiple grids, and predicts the bounding box and the probability of the category for each grid, and finally determines the final detection result through non-maximum suppression (NMS).

[0030] SSD: The full name is "Single Shot MultiBox Detector", which is a multi-box prediction real-time object detection algorithm. It predicts bounding boxes at multiple scales to detect objects of different sizes. SSD directly generates a series of default boxes (or anchor boxes) of fixed size and aspect ratio on the feature map of the image, and predicts whether these boxes contain the object, as well as the category and position offset of the object.

[0031] R-CNN: The full name is "Region CNN", which means regional convolutional neural network. R-CNN is an object detection algorithm based on candidate regions. It is divided into two main stages: candidate region generation and region classification. First, R-CNN uses a selective search algorithm to generate candidate regions in the image, that is, rectangular boxes that may contain objects. Then, convolutional neural network (CNN) feature extraction is performed on each candidate region, and classification is performed through support vector machine (SVM). Finally, bounding box regression is used to refine the location of the candidate region.

[0032] In this solution, the YOLOv5 algorithm is used. Figure 5 As shown in the figure, the network structure of YOLOv5 mainly consists of two parts: Backbone and Head prediction head. The backbone part is composed of CBS, C3 and fast pyramid pooling layers. The CBS module consists of convolution, batch normalization and activation function. The C3 module consists of CBS, tensor concatenation and tensor addition operations. This part extracts information from the image and uses the image feature information for the subsequent network. The prediction head part contains three parts: CSP structure, loss function and non-maximum suppression. The CSP structure consists of convolution layer and residual structure. This part better integrates and extracts the feature information given by the backbone part, thereby improving the performance of the network.

[0033] It should be noted that the collected information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for display, data for analysis, etc.) involved in this application are information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of relevant data are in compliance with relevant laws, regulations and standards, necessary confidentiality measures are taken, and public order and good customs are not violated, and corresponding operation entrances are provided for users to choose to authorize or refuse. For example, an interface is set up between this system and relevant users or institutions to provide users with corresponding operation entrances for users to choose to agree or refuse the results of automated decision-making; if the user chooses to refuse, the expert decision-making process will be entered.

[0034] Example 1

[0035] According to an embodiment of the present application, an embodiment of a target detection method is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0036] The method embodiment provided in the first embodiment of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 FIG. 1 shows a hardware structure block diagram of a computer terminal (or mobile device) for implementing a target detection method. Figure 1 As shown, the computer terminal 10 (or mobile device) may include one or more (102a, 102b, ..., 102n are used to illustrate) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0037] It should be noted that the one or more processors 102 and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10 (or mobile device). As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0038] The memory 104 can be used to store software programs and modules of application software, such as program instructions / data storage devices corresponding to the target detection method in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, realizing the above-mentioned target detection method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0039] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0040] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 (or mobile device).

[0041] Under the above operating environment, this application provides Figure 2 The target detection method shown. Figure 2 It is a flowchart of the target detection method according to Example 1 of the present application.

[0042] Step S101, obtaining image information of a target to be detected and a target detection data set;

[0043] The image information includes a first image, the target detection data set includes a second image, and the type of target object included in the first image is the same as the type of target object included in the second image.

[0044] Optionally, the image information refers to a specific image that needs to be detected, that is, the first image, and the target detection dataset contains a large number of images used to train and optimize the target detection algorithm, which are called second images. The target object type refers to the specific object category to be identified and located in the image, such as people, cars, trees, etc. The types of target objects contained in the first image and the second image must be the same, which means that the features and patterns learned by the model during the training process can also be found in the first image of the actual application. In this way, the model can successfully apply the detection capabilities it has acquired on the target detection dataset (second image) to the image information (first image) to achieve accurate detection. If the target object type does not match, the model may not be able to recognize the target in the first image, resulting in detection failure.

[0045] Step S102, determining first weight information based on the target detection data set, where the first weight information at least includes true values ​​of spatial attention weights corresponding to different regions of the second image;

[0046] Optionally, the first weight information is a set of spatial attention weights generated by processing the second image in the target data set. For each second image, the generated heat map is regarded as the true value of the spatial attention weight of the second image, and the set of true values ​​of all second images becomes the first weight information. The first weight information is used to supervise the output of the spatial attention module during the model training process, prompting the model to learn an attention distribution that can effectively enhance the feature representation of the target object.

[0047] Step S103: determining a target detection result based on the first weight information and the image information.

[0048] Among them, the target detection result includes at least one of the following: a bounding box prediction result, an object category classification result, and a confidence result.

[0049] Optionally, the bounding box prediction result refers to the boundary position of the target object predicted by the model in the image; the object category classification result is the model's judgment of which category the target object belongs to (such as people, cars, dogs, etc.); the confidence result is the model's confidence in the prediction result, usually a probability value between 0 and 1, and the larger the value, the more confident the model is.

[0050] Optionally, in a specific embodiment, assume that we are developing a system based on the improvement of YOLOv5 for identifying faces in a bank hall. First, obtain a target detection dataset, which contains a large number of images with detailed face annotations in bank scenes for model training. Generate a heat map for the face annotations in the second image, superimpose the heat maps of all targets, calculate the true value of the spatial attention weights of different regions, and form the first weight information. The first image captured by the real-time surveillance camera of the bank hall is fed into the model. The model uses the network structure and parameters learned in the training phase, while considering the first weight information (the true value heat map of the spatial attention weight) to start analyzing the first image. Based on the image features and the first weight information, the bounding box position of the face in the first image is predicted. These predicted bounding boxes will be compared with the actual face position. On the basis of determining the bounding boxes, the model further determines the category of the objects in these boxes to ensure that the boxes are indeed faces, not other objects or background elements. For each predicted bounding box, the model will give a confidence result indicating the probability that the model believes that the box is a face. This helps to screen out high-confidence detection results and reduce the false alarm rate. The final predicted face bounding box, the corresponding category (i.e. "face"), and the confidence of each prediction. This information can be used for subsequent processing, such as identity verification or surveillance analysis.

[0051] In an embodiment of the present application, a spatial attention weight quantization method is adopted to significantly improve the accuracy of target detection by determining the true value of the spatial attention weight corresponding to different regions based on the target detection data set, thereby achieving the purpose of more accurately focusing on the target object when detecting the first image, thereby achieving the technical effect of more effectively identifying and locating the target object when detecting images to be detected with complex and changeable backgrounds, and improving the overall detection performance, thereby solving the technical problem that the target detection algorithm is prone to false detection and missed detection under complex background conditions.

[0052] Optionally, in the target detection method provided in the embodiment of the present application, the target detection result is determined based on the first weight information and the image information, including: inputting the image information and the first weight information into a part of the backbone network in the target detection model for feature extraction to obtain output feature information, wherein the backbone network also includes a feature pyramid branch network; inputting the output feature information into the feature pyramid branch network to generate second weight information, the second weight information at least including the predicted value of the spatial attention weights of different regions of the first image; inputting the second weight information into the remaining part of the backbone network to obtain the target detection result.

[0053] Optionally, Figure 4This is the network structure diagram of the improved target detection algorithm based on YOLOv5. The image information and the first weight information (i.e., the true value heat map of the spatial attention weight) are input into the backbone network (Backbone) of the target detection model for feature extraction. The backbone network (Backbone) contains a series of convolutional layers (such as CBS modules), residual blocks (such as C3 modules) and a fast pyramid pooling layer (SPPF). CBS module: performs convolution, batch normalization, and activation function operations to extract the basic features of the first image. C3 module: Fusion of feature information at different levels through tensor concatenation and addition operations. SPPF layer: Pooling of the extracted features to enhance the model's ability to detect targets of different scales. The feature information output by the backbone network (Backbone) is passed to the feature pyramid branch network (FPN_b1) through the CBS module of the feature pyramid branch network (FPN_b1) for convolution operations and upsampling (Upsample) to generate the second weight information for different regions of the first image, i.e., the predicted value of the spatial attention weight, as shown in Figure 1. Figure 4 The generated second weight information is fed back to the feature map at a specific position in the backbone network in the form of a spatial attention weight prediction value, such as Figure 4 The second weight information is input into the remaining backbone network, and the bounding box prediction result, object category classification result and confidence result are obtained through the Head prediction head.

[0054] For example, a monitoring system needs to detect the face of a user in front of an ATM machine in real time for identity authentication. Real-time image information is obtained from the ATM monitoring camera, and a pre-trained model is used to generate first weight information that matches the image. The first weight information and image information are sent to the backbone network, and features of the image information are extracted through multiple CBS and C3 modules. The extracted feature information is input into the feature pyramid branch network to generate second weight information (prediction of spatial attention weight) for the user's facial position, and the second weight information is applied to the remaining backbone network to enhance facial area features and improve detection accuracy. Finally, the Head prediction head uses these enhanced feature maps to output accurate bounding boxes, categories (i.e., "face"), and high-confidence detection results of the user's face.

[0055] By adding the spatial attention module of the feature pyramid branch network to the backbone network, the target detection performance in complex backgrounds can be significantly improved, ensuring that the target objects can be accurately located and identified even in environments with cluttered background information.

[0056] Optionally, in the target detection method provided in an embodiment of the present application, determining the first weight information based on the target detection data set includes: determining the polyline label information based on the target detection data set, the polyline label information including the brief central structure morphology of the target object in the target detection data set; mapping the polyline labels contained in the polyline label information into continuous points to obtain continuous point information; and performing Gaussian blur on the continuous point information to obtain the first weight information.

[0057] Alternatively, if Figure 3 As shown in the S01 stage of the , image annotation software is used to carefully annotate each image in the target detection dataset. In addition to the conventional target bounding box, the brief central structure of the target object is additionally annotated. This central structure can be represented by a polyline, which can better depict the outline and key parts of the target object. For example, in face detection, the polyline may depict the connection line of feature points such as eyes, nose and mouth. The program reads the endpoint coordinates of each polyline, and then generates a coordinate for each point on the polyline path, so that the polyline is discretized into a series of continuous points. The generated continuous point information is Gaussian blurred to generate the spatial attention weight truth heat map of the target object. High-weight areas (key parts of the target object) will receive more attention and processing power from the model, while low-weight areas (background or irrelevant areas) will be weakened. Gaussian blurring spreads the weight value of each point to the surrounding area by applying a Gaussian kernel function to each point, thereby forming a smooth weight distribution. The degree of Gaussian blur (i.e., standard deviation σ) will affect the sharpness of the heat map. The smaller the standard deviation, the more concentrated the distribution of weight values ​​on the heat map, and the more it can highlight the key areas of the target object.

[0058] Specifically, the Gaussian blur conversion formula in the two-dimensional plane is used:

[0059]

[0060] Where x0, y0 represent the position coordinates of the pixel point that is being Gaussian blurred; c0 is the element value in the pixel point; x, y represent the position coordinates of other pixels in the current training set image; σ represents the standard deviation of the normal distribution. The calculation method of σ is:

[0061]

[0062] Among them l w Indicates the width information value in the YOLO format label file corresponding to the read true value bounding box; k represents the slope of the line segment on the polyline label corresponding to the target object. The weighted sum is calculated according to the length of the line segment, and the calculation formula of k is:

[0063]

[0064] Among them l i Indicates the length of each line segment on the polyline label corresponding to the target object in the image, k li Represents the slope of each line segment on the polyline, N l Represents the total number of segments on the polyline.

[0065] Optionally, in the monitoring scenario of ATM machines, the target detection dataset contains a large number of face images at different angles and under different lighting conditions, each image is annotated with a polyline, depicting the brief central structure of the face (such as the line connecting the eyes, nose and mouth). By converting the polyline into continuous points and applying Gaussian blur to generate a spatial attention weighted true value heat map, this process helps the model learn to focus on key facial features in a complex banking background during training, and accurately identify and locate each face even when the distance between customers is close and the background environment is complex, reducing false positives and false negatives, and improving overall detection efficiency and accuracy.

[0066] Through line annotation and Gaussian blur, the first weight information can highlight the central structural features of the target object, enabling the model to lock these key areas more quickly during training and detection. Even in images with complex background information, it can accurately identify the target object, reduce false detections and missed detections, and will not be disturbed by other objects in the background.

[0067] Optionally, determining the polyline label information based on the target detection data set includes: annotating at least one target object in the target detection data set with a polyline to obtain the polyline label information.

[0068] Optionally, unlike traditional bounding box annotation, polyline annotation focuses more on the central structure or outline of the target object, and depicts the key features of the target through a series of points or line segments. The model is provided with internal features and contour information of the target object. This polyline label information is richer and more detailed than simple bounding box annotation, which can help the model better understand the structure of the target object during learning, especially when dealing with objects with irregular shapes or complex structures. The polyline annotation effect is more significant. There may be more than one target object on the second image in the target detection dataset. Polyline annotation can be used to annotate target objects in different areas to improve detection accuracy.

[0069] Optionally, before inputting the image information and the first weight information into a partial backbone network in the target detection model for feature extraction, the method also includes: obtaining a true value heat map, wherein the true value heat map is determined by the first weight information; performing channel splicing on the true value heat map and the first image to obtain a spliced ​​image; performing data enhancement processing on the spliced ​​image to obtain an enhanced image; splitting the enhanced image into an enhanced true value heat map and an enhanced first image after channel splitting; determining the enhanced true value heat map as the first weight information; and determining the enhanced first image as image information.

[0070] Alternatively, if Figure 3 As shown, first, according to the line label information in the target detection data set, the first weight information is generated by Gaussian blur processing to obtain the true value heat map. This heat map contains the weight distribution information of the central structural morphology of the target object, and the area with higher weight corresponds to the key part of the target object. Next, the generated true value heat map is channel-joined with the original first image to form a four-channel spliced ​​image. This splicing enables the model to learn the visual features of the image and the structural information of the target object at the same time during training. Then, the spliced ​​image is subjected to data enhancement processing, including but not limited to rotation, translation, scaling, flipping, color transformation, etc. Data enhancement can increase the diversity of training data, help the model learn a wider range of image changes, and improve its robustness under different conditions. The processed enhanced image is split into the enhanced three-channel first image and the enhanced single-channel true value heat map through the channel splitting operation. This process ensures the independence of the information of the first image and the true value heat map information, so that the model can process and fuse these two types of information separately in the feature extraction stage. Thereby, the enhanced true value heat map is determined to be the first weight information; the enhanced first image is determined to be the image information.

[0071] Specifically, we are using the YOLOv5 model for face detection. The target detection dataset contains many images in bank scenes, in which the faces of people may change due to factors such as angle, lighting, and occlusion. Use image annotation software to annotate the faces in the original image with polylines, including eye sockets, nose bridge, and mouth contours. Then, these polylines are converted into true value heat maps through the Gaussian blur algorithm, and the generated true value heat map is channel-joined with the original image to form a four-channel image. Then, the four-channel image is subjected to data enhancement operations such as rotation, scaling, and color adjustment to generate multiple versions of the image. The data-enhanced four-channel image is split into an enhanced original image and an enhanced true value heat map. The enhanced original image is input into the YOLOv5 backbone network as image information for feature extraction, and the enhanced true value heat map is used as the first weight information to guide the model's attention allocation during the feature extraction process. The YOLOv5 model can learn the features of the face under different conditions from the enhanced image, and at the same time strengthen the recognition ability of the key parts of the face through the true value heat map, significantly improving the accuracy of face detection in banking scenarios. This improvement is not only applicable to face recognition, but can also be extended to other target detection tasks, such as hand movement recognition and important object detection in banking scenarios, providing more effective and accurate technical support for bank security monitoring and user identification.

[0072] By performing data enhancement processing on images, the diversity of the model's training samples can be increased and the model's generalization ability can be improved. This method can significantly improve the detection effect.

[0073] Optionally, inputting the second weight information into the remaining part of the backbone network to obtain the target detection result includes: inputting the second weight information into the remaining part of the backbone network for forward propagation to obtain a feature map; inputting the feature map into the head network of the target detection model to obtain the target detection result.

[0074] Specifically, the second weight information is generated by the spatial attention module in the feature pyramid network branch (FPN), which is input into the remaining backbone network and propagated forward to the SPPF. The SPPF layer helps to further integrate multi-scale features to enhance the small target detection capability. After the SPPF layer, it will pass through the C3_2 module to extract deeper features and generate feature maps. The optimized feature maps are finally sent to the Head prediction head to generate target detection results, including target category, location, confidence and other information.

[0075] By incorporating the second weight information into the feature map and optimizing the feature map, the Head prediction head can more accurately locate and classify targets, thereby improving the efficiency and accuracy of detection.

[0076] Optionally, after inputting the second weight information into the remaining part of the backbone network, the method also includes: constructing a loss function based on the first weight information and the second weight information; determining the loss parameters in this target detection process based on the loss function; and adjusting the network model parameters of the feature pyramid branch network based on the loss parameter feedback.

[0077] Specifically, Figure 3 As shown in the S03 section of , the construction of the loss function needs to combine the first weight information and the second weight information. Based on the loss function, we need to use an optimization algorithm (such as stochastic gradient descent SGD or Adam, etc.) to determine the loss parameters in this target detection process. The loss parameter reflects the degree of difference between the model prediction result and the true value label, and is the basis for adjusting the model parameters. When calculating the loss parameter, the model will compare the prediction result with the true value label, and combine the weighting of the first weight information and the second weight information, and determine the loss parameter of each prediction instance according to the loss calculation rules defined in the loss function, such as MSE (mean square error) or Cross-Entropy (cross entropy). After the loss parameter is determined, the loss gradient will be fed back to the parameters of each layer of the network model through the back propagation algorithm. In particular, in the feature pyramid branch network part, through the gradient information of the loss parameter, the feature pyramid branch network can learn how to generate a feature heat map that is more consistent with the spatial attention weight prediction value of the second weight information, thereby improving the accuracy of target detection.

[0078] This solution uses MSE as the empirical loss of the model and the performance indicator of the algorithm. The loss value L of the heat map hmap The calculation method is:

[0079]

[0080] Where N represents the number of images in a batch (target detection dataset) during network training; y represents the true value heat map of all images in a batch, y' represents the single-channel predicted spatial attention weight feature heat map of all images in a batch; yi represents the true value heat map of the i-th image in a batch, yi' represents the single-channel predicted spatial attention weight feature heat map of the i-th image in a batch.

[0081] Assume that the feature map of the YOLOv5 network at the returned position is x. In order to increase the weight, the single-channel predicted spatial attention weight feature heat map y' is returned to these positions and interacts with the feature map in the following way:

[0082] x'=x*(1+y')

[0083] Among them, x' is the feature map after adding the spatial attention weight, and is sent to the YOLOv5 subsequent network for further training.

[0084] By constructing a loss function, the model will calculate the difference between the predicted bounding box coordinates and category probabilities and the true labels, and at the same time evaluate the gap between the second weight information and the first weight information. Through continuous optimization of parameters, the accuracy and stability of detection can be improved.

[0085] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0086] Example 2

[0087] The present application also provides a target detection device. It should be noted that the target detection device of the present application can be used to execute the target detection method provided by the present application. The target detection device provided by the present application is introduced below.

[0088] According to an embodiment of the present application, a device for implementing the above target detection method is also provided, such as Figure 7 As shown, the device comprises:

[0089] An acquisition module 200 is used to acquire image information of a target to be detected and a target detection data set, wherein the image information includes a first image, the target detection data set includes a second image, and the target object type included in the first image is the same as the target object type included in the second image;

[0090] A first determination module 202, configured to determine first weight information based on the target detection data set, the first weight information at least including true values ​​of spatial attention weights corresponding to different regions of the second image;

[0091] The second determination module 204 is used to determine the target detection result based on the first weight information and the image information, wherein the target detection result includes at least one of the following: a bounding box prediction result, an object category classification result, and a confidence result.

[0092] The target detection device provided in the embodiment of the present application significantly improves the accuracy of target detection by weighting the spatial attention and determining the true values ​​of the spatial attention weights corresponding to different regions based on the target detection data set, thereby achieving the purpose of more accurately focusing on the target object when detecting the first image, thereby achieving the technical effect of more effectively identifying and locating the target object when detecting the image to be detected with a complex and changeable background, and improving the overall detection performance, thereby solving the technical problem that the target detection algorithm is prone to false detection and missed detection under complex background conditions.

[0093] Optionally, in the target detection device provided in the embodiment of the present application, the target detection result is determined based on the first weight information and the image information, including: inputting the image information and the first weight information into a part of the backbone network in the target detection model for feature extraction to obtain output feature information, wherein the backbone network also includes a feature pyramid branch network; inputting the output feature information into the feature pyramid branch network to generate second weight information, the second weight information at least including the predicted value of the spatial attention weights of different regions of the first image; inputting the second weight information into the remaining part of the backbone network to obtain the target detection result.

[0094] Optionally, in the target detection device provided in the embodiment of the present application, determining the first weight information based on the target detection data set includes: determining the polyline label information based on the target detection data set, the polyline label information including the brief central structure morphology of the target object in the target detection data set; mapping the polyline labels contained in the polyline label information into continuous points to obtain continuous point information; and performing Gaussian blur on the continuous point information to obtain the first weight information.

[0095] Optionally, in the target detection device provided in an embodiment of the present application, determining the broken line label information based on the target detection data set includes: annotating at least one target object in the target detection data set with a broken line to obtain the broken line label information.

[0096] Optionally, in the target detection device provided in the embodiment of the present application, before the image information and the first weight information are input into the backbone network of the part in the target detection model for feature extraction, the method also includes: obtaining a true value heat map, the true value heat map is determined by the first weight information; channel-splicing the true value heat map and the first image to obtain a spliced ​​image; performing data enhancement processing on the spliced ​​image to obtain an enhanced image; splitting the enhanced image into an enhanced true value heat map and an enhanced first image after channel splitting; determining the enhanced true value heat map as the first weight information; and determining the enhanced first image as image information.

[0097] Optionally, in the target detection device provided in the embodiment of the present application, the second weight information is input into the remaining part of the backbone network to obtain the target detection result, including: inputting the second weight information into the remaining part of the backbone network for forward propagation to obtain a feature map; inputting the feature map into the head network of the target detection model to obtain the target detection result.

[0098] Optionally, in the target detection device provided in the embodiment of the present application, after the second weight information is input into the remaining part of the backbone network, the method also includes: constructing a loss function based on the first weight information and the second weight information; determining the loss parameters in this target detection process based on the loss function; and adjusting the network model parameters of the feature pyramid branch network based on the loss parameter feedback.

[0099] It should be noted that the acquisition module 200, the first determination module 202, and the second determination module 204 correspond to steps S101 to S103 in Example 1, and the examples and application scenarios implemented by each module and the corresponding steps are the same, but are not limited to the contents disclosed in the above-mentioned Example 1. It should be noted that the above-mentioned modules or units may be hardware components or software components stored in a memory (e.g., memory 104) and processed by one or more processors (e.g., processors 102a, 102b, ..., 102n), and the above-mentioned modules may also be part of the device and may be run in the computer terminal 10 provided in Example 1.

[0100] Example 3

[0101] An embodiment of the present application may provide an electronic device, Figure 6 is a structural block diagram of an electronic device according to an embodiment of the present application. Figure 6 As shown, the electronic device may include: one or more ( Figure 6 (only one is shown) processor 1002, memory 1004, storage controller, and peripheral interface, wherein the peripheral interface is connected to the radio frequency module, audio module and display.

[0102] Among them, the memory can be used to store software programs and modules, such as program instructions / modules corresponding to the methods and devices in the embodiments of the present application, and the processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, realizing the above-mentioned method. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0103] The processor may call the information and application program stored in the memory through the transmission device to perform the following steps:

[0104] Step S101, obtaining image information of a target to be detected and a target detection data set;

[0105] The image information includes a first image, the target detection data set includes a second image, and the type of target object included in the first image is the same as the type of target object included in the second image.

[0106] Step S102, determining first weight information based on the target detection data set, where the first weight information at least includes true values ​​of spatial attention weights corresponding to different regions of the second image;

[0107] Step S103: determining a target detection result based on the first weight information and the image information.

[0108] Among them, the target detection result includes at least one of the following: a bounding box prediction result, an object category classification result, and a confidence result.

[0109] The processor can also call the information and application stored in the memory through the transmission device to perform the following steps: input the image information and the first weight information into the backbone network of the part in the target detection model for feature extraction to obtain output feature information, wherein the backbone network also includes a feature pyramid branch network; input the output feature information into the feature pyramid branch network to generate second weight information, and the second weight information includes at least the predicted value of the spatial attention weight of different areas of the first image; input the second weight information into the remaining backbone network to obtain the target detection result.

[0110] The processor can also call the information and application programs stored in the memory through the transmission device to perform the following steps: determine the broken line label information based on the target detection data set, the broken line label information includes the brief central structure morphology of the target object in the target detection data set; map the broken line labels contained in the broken line label information into continuous points to obtain continuous point information; perform Gaussian blur on the continuous point information to obtain the first weight information.

[0111] The processor may also call the information and application program stored in the memory through the transmission device to perform the following steps: annotating at least one target object in the target detection data set with a broken line to obtain broken line label information.

[0112] The processor can also call the information and application programs stored in the memory through the transmission device to perform the following steps: obtain a true value heat map, which is determined by the first weight information; perform channel splicing on the true value heat map and the first image to obtain a spliced ​​image; perform data enhancement processing on the spliced ​​image to obtain an enhanced image; split the enhanced image into an enhanced true value heat map and an enhanced first image after channel splitting; determine the enhanced true value heat map as the first weight information; determine the enhanced first image as image information.

[0113] The processor can also call the information and application programs stored in the memory through the transmission device to perform the following steps: input the second weight information into the remaining backbone network for forward propagation to obtain a feature map; input the feature map into the head network of the target detection model to obtain the target detection result.

[0114] The processor can also call the information and application programs stored in the memory through the transmission device to perform the following steps: construct a loss function based on the first weight information and the second weight information; determine the loss parameters in the current target detection process based on the loss function; and adjust the network model parameters of the feature pyramid branch network based on the loss parameter feedback.

[0115] In an embodiment of the present application, a spatial attention weight quantization method is adopted to significantly improve the accuracy of target detection by determining the true value of the spatial attention weight corresponding to different regions based on the target detection data set, thereby achieving the purpose of more accurately focusing on the target object when detecting the first image, thereby achieving the technical effect of more effectively identifying and locating the target object when detecting images to be detected with complex and changeable backgrounds, and improving the overall detection performance, thereby solving the technical problem that the target detection algorithm is prone to false detection and missed detection under complex background conditions.

[0116] It can be understood by those skilled in the art that Figure 6 The structure shown is for illustration only, and the electronic device may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (Mobile Internet Devices, MID), a PAD, or other terminal devices. Figure 6 The structure of the electronic device is not limited. Figure 6 More or fewer components (such as network interfaces, display devices, etc.) shown in, or having Figure 6 Different configurations are shown.

[0117] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0118] Example 4

[0119] According to another aspect of the present application, a computer-readable storage medium is provided, the computer-readable storage medium including a stored executable program, wherein when the executable program is executed, a device where the computer-readable storage medium is located is controlled to execute any one of the above methods.

[0120] The embodiment of the present application also provides a computer-readable storage medium. It should be noted that the computer-readable storage medium of the embodiment of the present application includes a stored executable program, wherein when the executable program is running, the device where the computer-readable storage medium is located can be used to execute the target detection method provided by the embodiment of the present application. The following introduces a computer-readable storage medium provided by the embodiment of the present application.

[0121] According to an embodiment of the present application, a computer-readable storage medium for implementing the above-mentioned target detection method is also provided, and the computer-readable storage medium executes the following steps:

[0122] Step S101, obtaining image information of a target to be detected and a target detection data set;

[0123] The image information includes a first image, the target detection data set includes a second image, and the type of target object included in the first image is the same as the type of target object included in the second image.

[0124] Step S102, determining first weight information based on the target detection data set, where the first weight information at least includes true values ​​of spatial attention weights corresponding to different regions of the second image;

[0125] Step S103: determining a target detection result based on the first weight information and the image information.

[0126] Among them, the target detection result includes at least one of the following: a bounding box prediction result, an object category classification result, and a confidence result.

[0127] Optionally, in the computer-readable storage medium provided in the embodiment of the present application, the following steps are performed: input the image information and the first weight information into a partial backbone network in the target detection model for feature extraction to obtain output feature information, wherein the backbone network also includes a feature pyramid branch network; input the output feature information into the feature pyramid branch network to generate second weight information, the second weight information at least including predicted values ​​of spatial attention weights of different regions of the first image; input the second weight information into the remaining backbone network to obtain a target detection result.

[0128] Optionally, in the computer-readable storage medium provided in the embodiment of the present application, the following steps are performed: determine the polyline label information based on the target detection data set, the polyline label information including the brief central structure morphology of the target object in the target detection data set; map the polyline labels contained in the polyline label information into continuous points to obtain continuous point information; perform Gaussian blur on the continuous point information to obtain first weight information.

[0129] Optionally, in the computer-readable storage medium provided in the embodiment of the present application, the following steps are performed: at least one target object in the target detection data set is annotated with a polyline to obtain polyline label information.

[0130] Optionally, in the computer-readable storage medium provided in the embodiment of the present application, the following steps are performed: obtain a true value heat map, the true value heat map is determined by the first weight information; channel-splicing the true value heat map and the first image to obtain a spliced ​​image; performing data enhancement processing on the spliced ​​image to obtain an enhanced image; splitting the enhanced image into an enhanced true value heat map and an enhanced first image after channel splitting; determining the enhanced true value heat map as the first weight information; determining the enhanced first image as image information.

[0131] Optionally, in the computer-readable storage medium provided in the embodiment of the present application, the following steps are performed: the second weight information is input into the remaining part of the backbone network for forward propagation to obtain a feature map; the feature map is input into the head network of the target detection model to obtain the target detection result.

[0132] Optionally, in the computer-readable storage medium provided in the embodiment of the present application, the following steps are performed: constructing a loss function based on the first weight information and the second weight information; determining the loss parameters in the current target detection process based on the loss function; and adjusting the network model parameters of the feature pyramid branch network based on the loss parameter feedback.

[0133] In an embodiment of the present application, a spatial attention weight quantization method is adopted to significantly improve the accuracy of target detection by determining the true value of the spatial attention weight corresponding to different regions based on the target detection data set, thereby achieving the purpose of more accurately focusing on the target object when detecting the first image, thereby achieving the technical effect of more effectively identifying and locating the target object when detecting images to be detected with complex and changeable backgrounds, and improving the overall detection performance, thereby solving the technical problem that the target detection algorithm is prone to false detection and missed detection under complex background conditions.

[0134] An embodiment of the present application further provides a computer program product. Optionally, in this embodiment, the computer program product includes computer instructions, and when the computer instructions are executed by a processor, the program implements any one of the above-mentioned target detection methods.

[0135] The steps of executing the target detection method by the computer program product are as follows:

[0136] Step S101, obtaining image information of a target to be detected and a target detection data set;

[0137] The image information includes a first image, the target detection data set includes a second image, and the type of target object included in the first image is the same as the type of target object included in the second image.

[0138] Step S102, determining first weight information based on the target detection data set, where the first weight information at least includes true values ​​of spatial attention weights corresponding to different regions of the second image;

[0139] Step S103: determining a target detection result based on the first weight information and the image information.

[0140] Among them, the target detection result includes at least one of the following: a bounding box prediction result, an object category classification result, and a confidence result.

[0141] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0142] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0143] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0144] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0145] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0146] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0147] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A target detection method, characterized in that: include: Acquire image information of a target to be detected and a target detection data set, wherein the image information includes a first image, the target detection data set includes a second image, and the target object type included in the first image is the same as the target object type included in the second image; Determine first weight information based on the target detection data set, where the first weight information at least includes true values ​​of spatial attention weights corresponding to different regions of the second image; A target detection result is determined based on the first weight information and the image information, wherein the target detection result includes at least one of the following: a bounding box prediction result, an object category classification result, and a confidence result.

2. The method according to claim 1, characterized in that Determining a target detection result based on the first weight information and the image information includes: Inputting the image information and the first weight information into a partial backbone network in the target detection model for feature extraction to obtain output feature information, wherein the backbone network also includes a feature pyramid branch network; Inputting the output feature information into the feature pyramid branch network to generate second weight information, wherein the second weight information at least includes predicted values ​​of spatial attention weights of different regions of the first image; The second weight information is input into the remaining part of the backbone network to obtain a target detection result.

3. The method according to claim 1, characterized in that Determining first weight information based on the target detection data set includes: Determine polyline label information based on the target detection data set, wherein the polyline label information includes a brief central structure form of the target object in the target detection data set; Mapping the polyline labels contained in the polyline label information into continuous points to obtain continuous point information; Gaussian blur is performed on the continuous point information to obtain the first weight information.

4. The method according to claim 3, characterized in that Determining the polyline label information based on the target detection data set includes: At least one target object in the target detection data set is annotated with a polyline to obtain the polyline label information.

5. The method according to claim 2, characterized in that: Before inputting the image information and the first weight information into a partial backbone network in the target detection model for feature extraction, the method further includes: Obtaining a true value heat map, where the true value heat map is determined by the first weight information; Perform channel splicing on the true value heat map and the first image to obtain a spliced ​​image; Performing data enhancement processing on the spliced ​​image to obtain an enhanced image; Splitting the enhanced image into an enhanced true value heat map and an enhanced first image through channel splitting; Determining the enhanced true value heat map as the first weight information; The enhanced first image is determined as the image information.

6. The method according to claim 2, characterized in that Inputting the second weight information into the remaining part of the backbone network to obtain the target detection result includes: Inputting the second weight information into the remaining part of the backbone network for forward propagation to obtain a feature map; The feature map is input into the head network of the target detection model to obtain the target detection result.

7. The method according to claim 2, characterized in that After inputting the second weight information into the remaining part of the backbone network, the method further includes: Constructing a loss function based on the first weight information and the second weight information; Determine the loss parameter in the target detection process based on the loss function; The network model parameters of the feature pyramid branch network are adjusted based on the loss parameter feedback.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored executable program, wherein when the executable program is executed, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 7.

9. An electronic device, characterized in that: include: A memory storing an executable program; A processor, configured to run the program, wherein the program executes the method according to any one of claims 1 to 7 when running.

10. A computer program product comprising computer instructions, characterized in that: When the computer instructions are executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images