Target detection method, target detection device, equipment and storage medium

Abstract

The embodiment of the invention provides a target detection method, a target detection device, equipment and a storage medium, and belongs to the technical field of financial science and technology. The method comprises the following steps: acquiring a training sample set, and adjusting a sample image based on a target size label and a preset target size to obtain an adjustment target and a sample adjustment image; dividing the adjustment target based on a preset target size to obtain an adjustment small target and an adjustment large target; detecting the sample adjustment image based on the initial detection model to obtain a first prediction target category; calculating a first small target loss contribution value based on the first prediction target category; determining a first small target area proportion based on the small target total area of the adjusted small target and the large target total area of the adjusted large target; and adjusting the initial detection model based on the first small target loss contribution value and the first small target area proportion to obtain a target detection model so as to perform target detection on the target image. According to the embodiment of the invention, the target detection efficiency can be improved on the basis of ensuring the detection precision.

Description

Technical Field

[0001] The present application relates to the field of financial technology, and in particular to a target detection method and a target detection device, equipment, and storage medium. Background Art

[0002] Object detection is an algorithm that identifies specific objects from images or videos and determines the location and size of these objects. Object detection includes small object detection, and the application scenarios of small object detection include remote sensing, autonomous driving, defect detection, etc. For example, in the video editing scenario of financial technology, it is necessary to edit according to the video content, which requires an accurate understanding of the various large and small objects in the video in order to edit the video more effectively.

[0003] At present, the related technology methods for detecting small targets in images usually improve the image resolution or crop the image for training to achieve detection of small targets in the image. However, increasing the image resolution will easily make the image larger, which will increase the amount of calculation for target detection, and cropping the image is equivalent to increasing the number of images, which will also increase the amount of calculation for target detection, thereby affecting the efficiency of target detection. Therefore, how to provide a method that can improve the efficiency of target detection while ensuring detection accuracy has become a technical problem that needs to be solved urgently. Summary of the invention

[0004] The main purpose of the embodiments of the present application is to propose a target detection method and a target detection device, equipment, and storage medium, which can improve the target detection efficiency while ensuring the detection accuracy.

[0005] To achieve the above object, a first aspect of an embodiment of the present application provides a target detection method, the method comprising:

[0006] Acquire a training sample set of a sample target, wherein the training sample set includes a sample image having the sample target and a sample target label of the sample image, wherein the sample target label includes a target size label and a sample target category label of the sample target in the sample image;

[0007] Performing image adjustment on the sample image based on the target size label and the preset target size to obtain an adjusted target and a sample adjusted image containing the adjusted target;

[0008] Dividing the adjustment target into small adjustment targets and large adjustment targets based on the preset target size;

[0009] Performing target detection on the sample adjustment image based on a preset initial detection model to obtain a first predicted target category of the adjustment target;

[0010] Calculate the loss contribution based on the first predicted target category and the sample target category label to obtain a first small target loss contribution value;

[0011] Determine the first small target area ratio based on the total area of ​​the small targets of the adjusted small targets and the total area of ​​the large targets of the adjusted large targets in the training sample set;

[0012] Adjusting model parameters of the initial detection model based on the first small target loss contribution value and the first small target area ratio to obtain a target detection model;

[0013] A target image is acquired, and target detection is performed on the target image based on the target detection model.

[0014] In some embodiments, the first predicted target category includes the predicted small target category of the adjusted small target and the predicted large target category of the adjusted large target, and the loss contribution calculation based on the first predicted target category and the sample target category to obtain the first small target loss contribution value includes:

[0015] Performing a category loss calculation based on the predicted small target category and the sample target category of the adjusted small target to obtain a small target category loss;

[0016] Performing a category loss calculation based on the predicted large target category and the sample target category of the adjusted large target to obtain a large target category loss;

[0017] Performing loss calculation on the small object category loss of the training sample set to obtain a total loss of the small object category;

[0018] Performing loss calculation on the large object category loss of the training sample set to obtain a total loss of the large object category;

[0019] The first small target loss contribution value is determined based on the total loss of the small target category and the total loss of the large target category.

[0020] In some embodiments, determining the first small target loss contribution value based on the total loss of the small target category and the total loss of the large target category includes:

[0021] Determine a loss weight based on the first small target area ratio, the loss weight comprising a first loss sub-weight of the total loss of the small target category and a second loss sub-weight of the total loss of the large target category;

[0022] Perform a weighted loss calculation based on the total loss of the small target category, the first loss sub-weight, the total loss of the large target category, and the second loss sub-weight to obtain a weighted total loss;

[0023] The loss proportion is calculated based on the total loss of the small target category, the first loss sub-weight and the weighted total loss to obtain the first small target loss contribution value.

[0024] In some embodiments, the image adjustment of the sample image based on the target size label and the preset target size to obtain the adjusted target and the sample adjusted image containing the adjusted target includes:

[0025] If the object size labels of the sample objects in the sample image are all larger than the preset object size, determining the maximum object size of the sample image based on the object size labels;

[0026] determining an image adjustment parameter based on the maximum target size and the preset target size;

[0027] Determining the number of stitched images based on the image adjustment parameters;

[0028] Selecting a candidate image from the training sample set based on the number of stitched images;

[0029] The sample image and the candidate image are subjected to image adjustment based on the image adjustment parameters to obtain the adjustment target and the sample adjusted image including the adjustment target.

[0030] In some embodiments, performing image adjustment on the sample image and the candidate image based on the image adjustment parameter to obtain the adjustment target and the sample adjusted image including the adjustment target includes:

[0031] If the sample image is the same as the candidate image, performing image enhancement on the candidate image to obtain a candidate enhanced image;

[0032] Performing image size adjustment on the sample image based on the image adjustment parameter to obtain a first adjustment sub-target and a first adjusted image including the first adjustment sub-target;

[0033] Based on the image adjustment parameter, the candidate enhanced image is resized to obtain a second adjustment sub-target and a second adjusted image including the second adjustment sub-target;

[0034] Determining the adjustment target based on the first adjustment sub-target and the second adjustment sub-target;

[0035] Performing image stitching on the first adjustment image and the second adjustment image to obtain a stitched image including the adjustment target, wherein the image size of the stitched image is the same as the image size of the sample image;

[0036] Perform image smoothing processing on the stitched image to obtain the sample adjusted image.

[0037] In some embodiments, before performing image adjustment on the sample image based on the target size label and the preset target size to obtain an adjustment target and a sample adjusted image containing the adjustment target, the method further includes:

[0038] If the sample image contains the sample target whose target size label is smaller than or equal to the preset target size, the sample target in the sample image is segmented based on the preset target size to obtain a sample small target and a sample large target;

[0039] Performing target detection on the sample image based on the initial detection model to obtain a second predicted target category of the sample target;

[0040] Calculate the loss contribution based on the second predicted target category and the sample target category label to obtain a second small target loss contribution value;

[0041] Determine the area ratio of the second small target based on the total area of ​​the small targets in the sample and the total area of ​​the large targets in the sample in the training sample set;

[0042] The model parameters of the initial detection model are adjusted based on the second small target loss contribution value and the second small target area ratio to obtain the target detection model.

[0043] In some embodiments, adjusting the model parameters of the initial detection model based on the first small target loss contribution value and the first small target area ratio to obtain the target detection model includes:

[0044] If the first small target loss contribution value is less than the target loss threshold, performing image enhancement on the sample adjusted image to obtain an enhanced adjusted image;

[0045] updating the training sample set based on the enhanced adjusted image;

[0046] Determine an updated first small target loss contribution value based on the updated training sample set, and compare the updated first small target loss contribution value with the target loss threshold value until the updated first small target loss contribution value is greater than or equal to the target loss threshold value;

[0047] The model parameters of the initial detection model are adjusted based on the updated first small target loss contribution value and the first small target area ratio to obtain the target detection model.

[0048] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application provides a target detection device, the device comprising:

[0049] An acquisition module, configured to acquire a training sample set of a sample target, wherein the training sample set includes a sample image having the sample target and a sample target label of the sample image, wherein the sample target label includes a target size label and a sample target category label of the sample target in the sample image;

[0050] An adjustment module, configured to perform image adjustment on the sample image based on the target size label and the preset target size, to obtain an adjustment target and a sample adjusted image containing the adjustment target;

[0051] A division module, used for dividing the adjustment target based on the preset target size to obtain a small adjustment target and a large adjustment target;

[0052] A first detection module, configured to perform target detection on the sample adjustment image based on a preset initial detection model to obtain a first predicted target category of the adjustment target;

[0053] A calculation module, configured to calculate a loss contribution based on the first predicted target category and the sample target category label to obtain a first small target loss contribution value;

[0054] A determination module, configured to determine a first small target area ratio based on a total area of ​​small targets of the adjusted small targets and a total area of ​​large targets of the adjusted large targets in the training sample set;

[0055] A training module, configured to adjust model parameters of the initial detection model based on the first small target loss contribution value and the first small target area ratio to obtain a target detection model;

[0056] The second detection module is used to acquire a target image and perform target detection on the target image based on the target detection model.

[0057] To achieve the above objectives, a third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method described in the first aspect is implemented.

[0058] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect is implemented.

[0059] The target detection method, target detection device, equipment, and storage medium proposed in the embodiments of the present application train the target detection model by combining the small target loss contribution value and the small target area ratio, and improve the accuracy of the model in detecting small targets, thereby improving the target detection efficiency while ensuring the detection accuracy. Specifically, by obtaining a training sample set of sample targets, the training sample set includes a sample image with sample targets and a sample target label of the image, and the sample target label includes a target size label and a sample target category label of the sample target in the sample image; further, based on the target size label and the preset target size, the sample image is adjusted to obtain an adjusted target and a sample adjusted image containing the adjusted target; based on the preset target size, the adjusted target is divided into an adjusted small target and an adjusted large target; further, based on a preset initial detection model, target detection is performed on the sample adjusted image to obtain a first predicted target category of the adjusted target; based on the first predicted target category and the sample target category label, loss contribution calculation is performed to obtain a first small target loss contribution value; based on the total area of ​​small targets of the adjusted small targets and the total area of ​​large targets of the adjusted large targets in the training sample set, the area ratio of the first small target is determined; further, based on the first small target loss contribution value and the first small target area ratio, model parameters of the initial detection model are adjusted to obtain a target detection model; finally, the target image is obtained, and target detection is performed on the target image based on the target detection model. Compared with the related technologies that train by improving image resolution or cropping images, the present application divides targets based on target size during model training, and combines the small target loss contribution value and the small target area ratio to improve the accuracy of small target detection by the target detection model. Therefore, the present application can better improve the target detection efficiency while ensuring detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 is a first flow chart of the target detection method provided in an embodiment of the present application;

[0061] Figure 2 yes Figure 1 A flowchart of step S120 in FIG.

[0062] Figure 3 is a schematic diagram of a sample image including a target size label provided in an embodiment of the present application;

[0063] Figure 4 yes Figure 2 A flowchart of step S250 in FIG.

[0064] Figure 5 is a schematic diagram of a process of image size adjustment provided by an embodiment of the present application;

[0065] Figure 6 is a schematic diagram of a process of image stitching provided by an embodiment of the present application;

[0066] Figure 7 yes Figure 1 A flowchart of step S150 in FIG.

[0067] Figure 8 yes Figure 7 A flowchart of step S750 in FIG.

[0068] Fig. 9 is a second flow chart of the target detection method provided in an embodiment of the present application;

[0069] Fig.10 yes Figure 1 A flowchart of step S170 in FIG.

[0070] Fig.11 Schematic diagram of a training process of a target detection model provided in an embodiment of the present application;

[0071] Fig.12 is a structural schematic diagram of a target detection device provided in an embodiment of the present application;

[0072] Fig.13 It is a hardware structure diagram of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0073] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0074] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0075] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0076] First, some nouns involved in this application are analyzed:

[0077] Artificial Intelligence (AI) is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. AI is a branch of computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can respond in a similar way to human intelligence. Research in this field includes robots, language recognition, image recognition, natural language processing and expert systems. AI can simulate the information process of human consciousness and thinking. AI is also a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0078] Computer Vision (CV) is a science that studies how to make machines "see". To put it more specifically, it refers to using cameras and computers to replace human eyes to identify, track and measure targets, and further perform image processing to make computer processing into images that are more suitable for human observation or transmission to instruments for detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to establish an artificial intelligence system that can obtain information from images or multi-dimensional data. Computer vision technology usually includes image processing, image recognition, image segmentation, image semantic understanding, image retrieval, video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous positioning and mapping, and other technologies, as well as common biometric recognition technologies such as face recognition and fingerprint recognition.

[0079] Object detection is an algorithm that identifies specific objects from images or videos and determines the location and size of these objects. Object detection includes small object detection, and the application scenarios of small object detection include remote sensing, autonomous driving, defect detection, etc. For example, in the video editing scene of financial technology, it is necessary to edit according to the video content, which requires an accurate understanding of the various large and small objects in the video in order to edit the video more effectively. Small object detection is an important direction in object detection. Compared with large objects, small object detection has certain difficulties. On the one hand, some small objects cannot be judged even by the naked eye, and the small scale and low resolution make the small objects have fewer visual features; on the other hand, because the proportion of small objects contained in each image is relatively small compared to medium and large objects, it will lead to sample imbalance problems, so it is difficult to achieve small object detection.

[0080] At present, the related technology methods for detecting small targets in images usually improve the image resolution or crop the image for training to achieve detection of small targets in the image. However, increasing the image resolution will easily make the image larger, which will increase the amount of calculation for target detection, and cropping the image is equivalent to increasing the number of images, which will also increase the amount of calculation for target detection, thereby affecting the efficiency of target detection. Therefore, how to provide a method that can improve the efficiency of target detection while ensuring detection accuracy has become a technical problem that needs to be solved urgently.

[0081] Based on this, the embodiments of the present application provide a target detection method and a target detection device, equipment, and storage medium, which can improve the target detection efficiency while ensuring the detection accuracy.

[0082] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0083] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0084] The target detection method provided in the embodiment of the present application relates to the field of artificial intelligence technology. The target detection method provided in the embodiment of the present application can be applied to a terminal, can also be applied to a server side, and can also be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, and can also be configured to provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (Content De l ivery Network, CDN) and cloud servers for basic cloud computing services such as big data and artificial intelligence platforms; the software can be an application that implements the target detection method, etc., but is not limited to the above forms.

[0085] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network personal computers (PC), minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0086] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on data related to the identity or characteristics of the object, such as object identity information, object behavior data, and object location information, the permission or consent of the object will be obtained first, and the collection, use, and processing of these data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present application needs to obtain sensitive personal information of the object, the separate permission or separate consent of the object will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the separate permission or separate consent of the object, the necessary object-related data for the normal operation of the embodiment of the present application will be obtained.

[0087] See also Figure 1 , Figure 1 is an optional flow chart of the target detection method provided in the embodiments of the present application. In some embodiments of the present application, Figure 1 The method may specifically include but is not limited to steps S110 to S180. Figure 1 These eight steps are introduced in detail.

[0088] Step S110, obtaining a training sample set of sample targets;

[0089] Step S120, performing image adjustment on the sample image based on the target size label and the preset target size to obtain an adjustment target and a sample adjusted image including the adjustment target;

[0090] Step S130, dividing the adjustment target based on the preset target size to obtain an adjustment small target and an adjustment large target;

[0091] Step S140, performing target detection on the sample adjustment image based on a preset initial detection model to obtain a first predicted target category of the adjustment target;

[0092] Step S150, performing loss contribution calculation based on the first predicted target category and the sample target category label to obtain a first small target loss contribution value;

[0093] Step S160, determining the first small target area ratio based on the total area of ​​the small targets adjusted in the training sample set and the total area of ​​the large targets adjusted in the large targets;

[0094] Step S170, adjusting the model parameters of the initial detection model based on the first small target loss contribution value and the first small target area ratio to obtain a target detection model;

[0095] Step S180, acquiring a target image, and performing target detection on the target image based on a target detection model.

[0096] In step S110 of some embodiments, the training sample set is a pre-constructed set for training the detection model of the present application, and the training sample set includes a sample image with a sample target and a sample target label of the sample image, and the sample target label includes a target size label and a sample target category label of the sample target in the sample image. For a sample image, the sample target is an object, person, building, etc. contained in the image. The sample target label is in the form of multiple labels, and the target size label is used to characterize the size of the sample target in the sample image. The target size label can be the size of a rectangular box determined based on the maximum boundary of the sample target. In this case, the target size label can be expressed in the form of a×b, and a represents the number of pixel units occupied by the height of the sample target in the sample image, and b represents the number of pixel units occupied by the width of the sample target in the sample image. The sample target category label is used to characterize the category of the sample target, such as an object, person, building, etc.

[0097] It should be noted that the training sample set may be a sample set of a current training batch, and the present application may train the model based on training sample sets of multiple batches.

[0098] In step S120 of some embodiments, for each sample image, at least one sample target may be included, and each sample target has a corresponding sample target label. Before training the model, the present application may determine the large and small targets contained in the sample image based on the target size label and the preset target size, so as to determine whether the sample image needs to be adjusted to better enable the model to detect targets of different sizes. Among them, the preset target size refers to a pre-set size for determining small targets. For example, if the small target refers to less than 32×32 pixels, the preset target size can be set to 32×32 pixels, or it can be adjusted according to actual needs without limitation.

[0099] See also Figure 2 , Figure 2 is an optional flowchart of step S120 provided in an embodiment of the present application. In some embodiments of the present application, step S120 may specifically include steps S210 to S250. Figure 2 These five steps are introduced in detail.

[0100] Step S210, if the target size labels of the sample targets in the sample image are all larger than the preset target size, determining the maximum target size of the sample image based on the target size labels;

[0101] Step S220, determining image adjustment parameters based on the maximum target size and the preset target size;

[0102] Step S230, determining the number of stitched images based on the image adjustment parameters;

[0103] Step S240, selecting candidate images from the training sample set based on the number of stitched images;

[0104] Step S250 , performing image adjustment on the sample image and the candidate image based on the image adjustment parameters to obtain an adjustment target and a sample adjusted image including the adjustment target.

[0105] In step S210 of some embodiments, if the target size labels of the sample targets in the sample image are greater than or equal to the preset target size, it means that the current sample image does not contain small targets. At this time, in order to enable the model to be trained for small target detection, the present application needs to adjust the sample image. The maximum target size refers to the maximum value of the target size label corresponding to the sample target contained in the current sample image. For example, the preset target size is set to 32×32 pixels, and the current sample image only contains target objects with a size of 64×64 pixels or less. At this time, the maximum target size can be determined to be 64×64 pixels.

[0106] For example, Figure 3 As shown, Figure 3 3 is a schematic diagram of a sample image including a target size label provided by an embodiment of the present application. The sample image 310 includes sample target A, sample target B, and sample target C. The target size label of sample target A is 128×128 pixels, and the target size labels of sample target B and sample target C are both 64×64 pixels. If the preset target size is set to 32×32 pixels, it can be determined that the target size labels of the sample targets contained in the current sample image 310 are all greater than or equal to the preset target size, that is, they are all large targets, and then it can be determined that the maximum target size of the sample image 310 is 128×128 pixels.

[0107] In step S220 of some embodiments, after determining the maximum target size and the preset target size, the maximum target size can be divided by the preset target size to determine the image adjustment parameters. The image adjustment parameters are used to characterize the scaling factor of the sample image. That is, when the sample image does not contain small targets, in order to enable the model to learn to detect small targets, the present application can determine the image adjustment parameters for adjusting the image size of the sample image so that the adjusted image contains small targets. For example, the maximum target size is 64×64 pixels, and the preset target size is 32×32 pixels. At this time, the image adjustment parameters may include reducing the height direction to half of the original size and reducing the width direction to half of the original size, so that the adjusted maximum target size meets the requirements of the preset target size corresponding to the small target, and then other targets must also meet the requirements, thereby effectively increasing the number of small targets in the sample image.

[0108] It should be noted that, since this application focuses on the detection of small targets, the image adjustment parameters usually represent the reduction ratio of the sample image.

[0109] In step S230 of some embodiments, since the sample image is adjusted based on the image adjustment parameters, the size of the sample image will change, thereby affecting the requirements of the subsequent model for the input image. Therefore, the present application will simultaneously splice the adjusted image with other images so that the spliced ​​image is the same as the original image, or the same as the required size of the model for the input image. For example, the image adjustment parameters may include reducing the height direction to half of the original size and reducing the width direction to half of the original size. At this time, the sample image is only one-fourth of the original size after adjustment. For another example, for a sample image that does not contain a small target of 32×32 pixels (but contains targets of 128×128 pixels and below), the height and width of the sample image can be reduced and adjusted to one-fourth of the original size. In order to reach the original size of the sample image when inputting the model, it is necessary to splice some other images, among which three images can be spliced ​​evenly, or two images of different sizes can be spliced ​​to reach the original size of the sample image, which is not limited here.

[0110] In step S240 of some embodiments, after the number of stitched images is determined, a candidate image can be selected from multiple sample images in the training sample set. The candidate image can be the same as or different from the sample image to be stitched, in order to increase the number of targets, and the total number of candidate images and sample images is the number of stitched images. In addition, the number of targets contained in the selected candidate image is not limited and can be randomly selected.

[0111] In step S250 of some embodiments, the sample image and the candidate image are further adjusted based on the image adjustment parameters to obtain an adjustment target and a sample adjustment image containing the adjustment target. The adjustment target refers to the target contained in the adjusted sample adjustment image. The sample adjustment image contains the sample image and the candidate image.

[0112] In the above steps S210 to S250, for sample images that do not contain small targets, the present application determines the image adjustment parameters for adjusting the sample images by using the maximum target size and the preset target size of the sample images, so that the number of small targets contained in the image may be increased to a greater extent. Further, the number of stitched images minus 1 candidate images is determined by the determined image adjustment parameters to perform image adjustment on the sample image and the candidate images, and obtain the adjusted target and the sample adjusted image containing the adjusted target, so that the accuracy of small target detection can be improved in the subsequent model training process.

[0113] See also Figure 4 , Figure 4is an optional flowchart of step S250 provided in an embodiment of the present application. In some embodiments of the present application, step S250 may specifically include steps S410 to S460. Figure 4 These six steps are introduced in detail.

[0114] Step S410, if the sample image and the candidate image are the same, perform image enhancement on the candidate image to obtain a candidate enhanced image;

[0115] Step S420, adjusting the image size of the sample image based on the image adjustment parameter to obtain a first adjustment sub-target and a first adjusted image including the first adjustment sub-target;

[0116] Step S430, adjusting the image size of the candidate enhanced image based on the image adjustment parameter to obtain a second adjustment sub-target and a second adjusted image including the second adjustment sub-target;

[0117] Step S440, determining an adjustment target based on the first adjustment sub-target and the second adjustment sub-target;

[0118] Step S450, stitching the first adjusted image and the second adjusted image to obtain a stitched image containing an adjustment target;

[0119] Step S460: performing image smoothing processing on the stitched image to obtain a sample adjusted image.

[0120] In step S410 of some embodiments, if the sample image and the candidate image are the same, in order to distinguish different images during stitching, the candidate image may be enhanced to obtain a candidate enhanced image. The image enhancement may include different noise addition operations, random flipping, brightness enhancement, etc., which are not specifically limited.

[0121] In step S420 of some embodiments, the sample image is resized based on the image adjustment parameter to obtain a first adjustment sub-target and a first adjusted image containing the first adjustment sub-target. Since the image adjustment parameter can scale the original sample image, the sample target contained therein will be adjusted at the same time to obtain the first adjustment sub-target. For example, Figure 5 As shown, Figure 5: is a schematic diagram of the image size adjustment process provided by an embodiment of the present application. If the image adjustment parameter indicates that the height and width pixel units of the original sample image 310 need to be reduced to half of the original, then the sample target A in the sample image 310 can be adjusted to the first adjustment sub-target A', the sample target B in the sample image 310 can be adjusted to the first adjustment sub-target B', and the sample target C in the sample image 310 can be adjusted to the first adjustment sub-target C', and the height and width pixel units of the adjusted first adjustment image are reduced to half of the original.

[0122] In step S430 of some embodiments, similar to step S420, the candidate enhanced image is resized based on the image adjustment parameter to obtain a second adjusted sub-target and a second adjusted image containing the second adjusted sub-target. The second adjusted sub-target refers to a target after scaling the target in the candidate enhanced image, and the second adjusted image refers to an image obtained after adjusting the candidate enhanced image.

[0123] In step S440 and step S450 of some embodiments, all adjusted adjustment targets can be determined based on the first adjustment sub-target and the second adjustment sub-target. Further, the first adjustment image and the second adjustment image are stitched to obtain a stitched image containing the adjustment target, and the image size of the stitched image is the same as the image size of the sample image. For example, Figure 6 As shown, Figure 6 : is a schematic diagram of the image stitching process provided by an embodiment of the present application. Among them, if the image adjustment parameter indicates that the height and width pixel units of the original sample image 310 need to be reduced to half of the original, it can be determined that three candidate enhanced images need to be stitched. Further, the sample image 310 is subjected to image size adjustment to obtain a first adjusted image 610, and the candidate enhanced image 620 is subjected to image size adjustment to obtain a second adjusted image 630. Afterwards, the first adjusted image 610 and the second adjusted image 630 are stitched to obtain a stitched image 640 containing the adjustment target.

[0124] It should be noted that the present application does not specifically limit the order in which the first adjusted image 610 and the second adjusted image 630 are spliced.

[0125] In step S460 of some embodiments, after obtaining the stitched image, the present application may perform image smoothing processing on the stitched image to obtain a sample adjusted image. The image smoothing processing may be a randomly selected median filter, a Gaussian filter, or other processing method to improve the smoothness of the stitched image, thereby making the sample adjusted image closer to the real image.

[0126] In other embodiments, step S250 may further specifically include: if the sample image and the candidate image are different, the candidate image may be enhanced, or the original candidate image may be used to perform subsequent steps S420 to S460, which will not be described in detail here.

[0127] In step S130 of some embodiments, after obtaining the sample adjustment image and the adjustment target, the number of small targets included in the training sample set of the current batch of the present application will increase, and the present application can first divide the adjustment target based on the preset target size to obtain the small adjustment target and the large adjustment target. The small adjustment target refers to the target whose size is less than or equal to the preset target size in the adjustment target. The large adjustment target refers to the target whose size is larger than the preset target size in the adjustment target.

[0128] It should be noted that the adjustment targets of the present application also include targets in other unchanged sample images in the training sample set. That is to say, adjusting the small target can represent targets in the training sample set including the sample adjustment image whose size is less than or equal to the preset target size, and adjusting the large target can represent targets in the training sample set including the sample adjustment image whose size is greater than the preset target size.

[0129] In step S140 of some embodiments, after obtaining the sample adjustment image, target detection can be performed on the sample adjustment image and other images in the training sample set based on a preset initial detection model to obtain a first predicted target category of the adjustment target. The first predicted target category includes a predicted small target category for adjusting a small target and a predicted large target category for adjusting a large target, and the predicted small target category and the predicted large target category at this time can be an object, a building, a person, etc.

[0130] In step S150 of some embodiments, further, a loss contribution calculation may be performed based on the first predicted target category and the corresponding sample target category label to obtain a first small target loss contribution value. The first small target loss contribution value is used to characterize the degree of loss contribution of the loss value determined based on the adjustment of the small target to the overall loss value.

[0131] See also Figure 7 , Figure 7 is an optional flowchart of step S150 provided in an embodiment of the present application. In some embodiments of the present application, step S150 may specifically include steps S710 to S750. Figure 7 These five steps are introduced in detail.

[0132] Step S710, performing category loss calculation based on the predicted small target category and the sample target category of the adjusted small target to obtain the small target category loss;

[0133] Step S720, performing category loss calculation based on the predicted large target category and the adjusted large target sample target category to obtain the large target category loss;

[0134] Step S730, calculating the loss of the small object category of the training sample set to obtain the total loss of the small object category;

[0135] Step S740, calculating the loss of the large object category of the training sample set to obtain the total loss of the large object category;

[0136] Step S750, determining a first small target loss contribution value based on the total loss of the small target category and the total loss of the large target category.

[0137] In step S710 and step S720 of some embodiments, the present application may perform corresponding category loss calculations based on the predicted small target category and the predicted large target category, respectively, to obtain the small target category loss and the large target category loss. The process of calculating the category loss may adopt cross-entropy loss, focal loss, high loss, etc., which are not specifically limited here.

[0138] In step S730 and step S740 of some embodiments, the model will summarize the category losses of all small targets in the entire training sample set, that is, the small target category losses corresponding to all small targets in the training sample set are summed up to obtain the total loss of the small target category. In this way, the overall performance of the model when processing small target data can be fully understood. At the same time, the model will also summarize the category losses of all large targets in the entire training sample set, that is, the large target category losses corresponding to all large targets in the training sample set are summed up to obtain the total loss of the large target category. In this way, the overall performance of the model when processing large target data can be fully understood, thereby improving the detection accuracy of the model for large and small targets.

[0139] In step S750 of some embodiments, further, the present application may determine the first small target loss contribution value based on the total loss of the small target category and the total loss of the large target category, and the process of determining the first small target loss contribution value is shown in the following formula 1:

[0140]

[0141] In formula 1, α s Indicates the first small target loss contribution value, L s represents the total loss of small target category, L oIt represents the total loss of the large target category, and the total loss of the large target category can represent other targets except small targets, so that the influence of the background on the model training can be excluded.

[0142] In the above embodiment, the present application can improve the detection accuracy of the model for large targets and small targets at the same time by comprehensively considering the total loss of small target categories and the total loss of large target categories. At the same time, by considering the contribution of the total loss of small target categories to the total loss of all target categories, the model's recognition ability for small targets can be enhanced.

[0143] In step S160 of some embodiments, in order to determine whether to perform image adjustment in the process of detecting model training, the present application may also set a specific threshold for judgment, namely, the first small target area ratio. Specifically, the first small target area ratio may be determined based on the total area of ​​small targets of the small targets adjusted in the training sample set and the total area of ​​large targets of the large targets adjusted. The small targets adjusted at this time refer to each small target included in the training sample set containing the sample adjustment image. The first small target area ratio is used to characterize the ratio of the image area occupied by all small targets in the training sample set to the total image area. For example, the training sample set after image adjustment includes target A1 (corresponding to target area S1), target A2 (corresponding to target area S2), target A3 (corresponding to target area S3), target A4 (corresponding to target area S4) and target A5 (corresponding to target area S5), and targets A1, A2 and A3 are small targets. At this time, the first small target area ratio = (S1+S2+S3) / (S1+S2+S3+S4+S5).

[0144] See also Figure 8 , Figure 8 8 is an optional flow chart of step S750 provided in an embodiment of the present application. In some embodiments of the present application, step S750 may specifically include steps S810 to S830. These three steps are described in detail below in conjunction with 8.

[0145] Step S810, determining a loss weight based on the first small target area ratio;

[0146] Step S820, performing a weighted loss calculation based on the total loss of the small target category, the first loss sub-weight, the total loss of the large target category, and the second loss sub-weight to obtain a weighted total loss;

[0147] Step S830, calculating the loss proportion based on the total loss of the small target category, the first loss sub-weight and the weighted total loss to obtain the first small target loss contribution value.

[0148] In step S810 of some embodiments, the present application may determine the loss weight based on the first small target area ratio. For example, if the first small target area ratio is less than the preset ratio threshold, it means that the number of small targets contained in the training sample set is small, which will affect the model's detection accuracy for small targets. In this way, a higher weight can be assigned to the total loss of the small target category to remind the model to better optimize in the direction of reducing the total loss of the small target category; conversely, a smaller weight is assigned to the total loss of the small target category, indicating that the model's current training on small targets is sufficient to remind the model to optimize in the direction of reducing the overall loss. The loss weight includes a first loss sub-weight of the total loss of the small target category and a second loss sub-weight of the total loss of the large target category.

[0149] In step S820 and step S830 of some embodiments, further, the process of calculating the first small target loss contribution value may also be shown in the following formula 2:

[0150]

[0151] In formula 2, α s represents the first small target loss contribution value, β1 represents the first loss sub-weight, β2 represents the second loss sub-weight, L s represents the total loss of small target category, L o Represents the total loss of large target categories.

[0152] In the above embodiment, when calculating the loss contribution value of the first small target, the present application can determine the corresponding loss weight based on the proportion of the area of ​​the first small target, which can better enhance the model's recognition of small targets.

[0153] See also Fig. 9 , Fig. 9 is another optional flow chart of the target detection method provided by the embodiment of the present application. In some embodiments of the present application, before step S120, the target detection method provided by the embodiment of the present application may further include steps S910 to S950. Fig. 9 These five steps are introduced in detail.

[0154] Step S910, if the sample image contains sample targets whose target size labels are smaller than or equal to the preset target size, target segmentation is performed on the sample targets in the sample image based on the preset target size to obtain sample small targets and sample large targets;

[0155] Step S920, performing target detection on the sample image based on the initial detection model to obtain a second predicted target category of the sample target;

[0156] Step S930, performing loss contribution calculation based on the second predicted target category and the sample target category label to obtain a second small target loss contribution value;

[0157] Step S940, determining the area proportion of the second small target based on the total area of ​​the small targets in the training sample set and the total area of ​​the large targets in the large targets;

[0158] Step S950, adjusting the model parameters of the initial detection model based on the second small target loss contribution value and the second small target area ratio to obtain a target detection model.

[0159] In steps S910 to S950 of some embodiments, if the sample image contains a sample target whose target size label is less than or equal to the preset target size, it means that the current sample image contains a small target. Then, the sample targets in the sample image can be directly divided based on the preset target size to obtain sample small targets and sample large targets, which is the same as the above step S130. Further, target detection is performed on the sample image based on the initial detection model to obtain a second predicted target category of the sample target, which is the same as the above step S140. Loss contribution calculation is performed based on the second predicted target category and the sample target category label to obtain a second small target loss contribution value, which is the same as the above step S150. Based on the total small target area of ​​the sample small targets in the training sample set and the total large target area of ​​the sample large targets, the second small target area ratio is determined, which is the same as the above step S160. Based on the second small target loss contribution value and the second small target area ratio, the model parameters of the initial detection model are adjusted to obtain a target detection model, which is the same as the above step S170, except that the parameters have changed. In order to save space, it will not be repeated.

[0160] In step S170 of some embodiments, the model parameters of the initial detection model are adjusted based on the first small target loss contribution value and the first small target area ratio to obtain a target detection model.

[0161] In the above embodiment, the present application determines whether the sample image contains a small target, and considers the comparative relationship between the first small target loss contribution value and the first small target area ratio to adjust the model parameters of the initial detection model. Compared with the related art by improving the image resolution or cropping the image for training, the present application does not increase the number of images in the process of model training, but first divides the target based on the target size, considers the target situation contained in the image itself, and combines the small target loss contribution value and the small target area ratio to improve the accuracy of the target detection model for small target detection. Therefore, the present application can better improve the efficiency of target detection while ensuring the detection accuracy.

[0162] It should be noted that the initial detection model used in this application can be a deep learning model, such as residual neural network (ResNet50), convolutional neural network (Convolutional Neural Network, CNN), regional convolutional neural network (Region-CNN, RCNN), yo lo series, etc., or, a machine learning model can also be used, without limitation.

[0163] See also Fig.10 , Fig.10 is an optional flowchart of step S170 provided in an embodiment of the present application. In some embodiments of the present application, step S170 may specifically include steps S1010 to S1040. Fig.10 These four steps are introduced in detail.

[0164] Step S1010, if the first small target loss contribution value is less than the target loss threshold, image enhancement is performed on the sample adjustment image to obtain an enhanced adjustment image;

[0165] Step S1020, updating the training sample set based on the enhanced adjusted image;

[0166] Step S1030, determining an updated first small target loss contribution value based on the updated training sample set, and comparing the updated first small target loss contribution value with the target loss threshold value until the updated first small target loss contribution value is greater than or equal to the target loss threshold value;

[0167] Step S1040, adjusting the model parameters of the initial detection model based on the updated first small target loss contribution value and the first small target area ratio to obtain a target detection model.

[0168] In step S1010 and step S1020 of some embodiments, if the first small target loss contribution value is less than the target loss threshold, it means that the total amount of small targets contained in the current training sample set is insufficient for model training. In this case, the sample adjustment image can be enhanced to obtain an enhanced adjustment image, and the training sample set can be updated based on the enhanced adjustment image. Wherein, updating the training sample set based on the enhanced adjustment image can be to replace the corresponding sample image or the sample image with a small number of small targets in the training sample set with the enhanced adjustment image, or the enhanced adjustment image can also be added to the training sample set to increase the number of samples contained in the training sample set.

[0169] In step S1030 of some embodiments, further, an updated first small target loss contribution value is determined based on the updated training sample set, and based on the comparison between the updated first small target loss contribution value and the target loss threshold, if the first small target loss contribution value is still smaller than the target loss threshold, the above step S120 can be performed on other sample images to increase the number of small targets included in the training sample set, and recalculate the first small target loss contribution value until the updated first small target loss contribution value is greater than or equal to the target loss threshold.

[0170] In step S1040 of some embodiments, after the present application completes training of the current batch of training sample sets, if it is detected that the target detection model has not reached the preset detection accuracy, the next batch of training sample sets may be used for training until the preset detection accuracy is reached.

[0171] It should be noted that during the inference stage of this application, there is no need to adjust the multi-scale test or other operations to adapt to this application, and there will be no impact on the inference speed.

[0172] For example, Fig.11 As shown, Fig.11 It is a schematic diagram of the training process of the target detection model provided by an embodiment of the present application. Among them, for a sample image, the present application can first detect whether the sample image contains a small target. If it does, the sample image can be directly input into the initial detection model; if it does not, the sample image is subjected to image adjustment through the above step S120 to obtain a sample adjusted image. Further, the first small target loss contribution value is obtained by calculating the total loss of the small target category and the total loss of the large target category. Further, the first small target loss contribution value and the first small target area ratio are compared. If the first small target loss contribution value is less than the first small target area ratio, the sample image is continued to be spliced ​​and adjusted, and the spliced ​​and adjusted sample image is input into the initial detection model for training; if the first small target loss contribution value is greater than or equal to the first small target area ratio, the current sample image is used to input into the initial detection model for training. Based on this, in order to solve the problem of insufficient supervision signals of small target objects in the sample set, the present application can use a data loader to dynamically generate spliced ​​images or conventional sample images to enrich small targets.

[0173] In step S180 of some embodiments, after training the target detection model, the present application can obtain the target image and perform target detection on the target image based on the target detection model. For example, in the video editing scene of financial technology, it is necessary to edit according to the video content, which requires an accurate understanding of the various large and small targets in the video in order to more effectively perform video editing. Each frame of the video can be input into the target detection model for target detection to determine the target contained in each frame. Furthermore, the operating object can modify, replace, cut, and other adjustments to the target.

[0174] It should be noted that the non-Company software tools or components appearing in the embodiments of the present application are merely examples and do not represent actual use.

[0175] A target detection method provided in an embodiment of the present application uses supervision of the loss contribution of small targets to dynamically enhance the data, and significantly improves the small target detection by designing a loss feedback-driven mechanism (i.e., by comparing the first small target loss contribution value and the first small target area ratio) and an image stitching method, which can enhance the target detection model's ability to detect small targets without affecting the reasoning speed. Compared to the related art by increasing the image resolution or performing image cropping training, the present application considers whether there are small targets in the image, whether the image is data enhanced separately before stitching, whether filtering operations are performed after stitching to increase smoothness, etc. Therefore, the present application can better improve the target detection efficiency while ensuring detection accuracy.

[0176] See also Fig.12 , Fig.12 : is a schematic diagram of the module structure of the target detection device provided in the embodiment of the present application. The device can implement the target detection method of the above embodiment. The device may specifically include:

[0177] An acquisition module 1210 is used to acquire a training sample set of sample targets, wherein the training sample set includes a sample image having the sample target and a sample target label of the sample image, wherein the sample target label includes a target size label and a sample target category label of the sample target in the sample image;

[0178] An adjustment module 1220, configured to perform image adjustment on the sample image based on the target size label and the preset target size, and obtain an adjustment target and a sample adjusted image including the adjustment target;

[0179] A division module 1230 is used to divide the adjustment target into small adjustment targets and large adjustment targets based on a preset target size;

[0180] A first detection module 1240 is used to perform target detection on the sample adjustment image based on a preset initial detection model to obtain a first predicted target category of the adjustment target;

[0181] A calculation module 1250 is used to calculate the loss contribution based on the first predicted target category and the sample target category label to obtain a first small target loss contribution value;

[0182] A determination module 1260 is used to determine the first small target area ratio based on the total area of ​​small targets adjusted in the training sample set and the total area of ​​large targets adjusted in the training sample set;

[0183] A training module 1270 is used to adjust the model parameters of the initial detection model based on the first small target loss contribution value and the first small target area ratio to obtain a target detection model;

[0184] The second detection module 1280 is used to acquire a target image and perform target detection on the target image based on a target detection model.

[0185] It should be noted that the target detection device of the embodiment of the present application is used to implement the target detection method of the above-mentioned embodiment. The target detection device of the embodiment of the present application corresponds to the above-mentioned target detection method. Please refer to the above-mentioned target detection method for the specific processing process, which will not be repeated here.

[0186] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above target detection method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0187] See also Fig.13 , Fig.13 The hardware structure of a computer device according to another embodiment is illustrated. The computer device includes:

[0188] The processor 1310 may be implemented by a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application.

[0189] The memory 1320 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1320 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1320, and the processor 1310 calls and executes the target detection method of the embodiment of this application;

[0190] Input / output interface 1330, used to implement information input and output;

[0191] The communication interface 1340 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WI FI, Bluetooth, etc.);

[0192] bus 1350 , which transmits information between the various components of the device (e.g., processor 1310 , memory 1320 , input / output interface 1330 , and communication interface 1340 );

[0193] The processor 1310 , the memory 1320 , the input / output interface 1330 , and the communication interface 1340 are connected to each other in communication within the device via a bus 1350 .

[0194] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned target detection method is implemented.

[0195] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0196] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0197] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0198] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0199] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0200] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0201] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0202] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0203] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0204] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0205] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes multiple instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store programs.

[0206] The preferred embodiments of the present invention are described with reference to the accompanying drawings, but the scope of the present invention is not limited by the above. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the present invention.

Claims

1. A target detection method, characterized in that: The method comprises: Acquire a training sample set of a sample target, wherein the training sample set includes a sample image having the sample target and a sample target label of the sample image, wherein the sample target label includes a target size label and a sample target category label of the sample target in the sample image; Performing image adjustment on the sample image based on the target size label and the preset target size to obtain an adjusted target and a sample adjusted image containing the adjusted target; Dividing the adjustment target into small adjustment targets and large adjustment targets based on the preset target size; Performing target detection on the sample adjustment image based on a preset initial detection model to obtain a first predicted target category of the adjustment target; Calculate the loss contribution based on the first predicted target category and the sample target category label to obtain a first small target loss contribution value; Determine the first small target area ratio based on the total area of ​​the small targets of the adjusted small targets and the total area of ​​the large targets of the adjusted large targets in the training sample set; Adjusting model parameters of the initial detection model based on the first small target loss contribution value and the first small target area ratio to obtain a target detection model; A target image is acquired, and target detection is performed on the target image based on the target detection model.

2. The method according to claim 1, characterized in that: The first predicted target category includes the predicted small target category of the adjusted small target and the predicted large target category of the adjusted large target, and the loss contribution calculation based on the first predicted target category and the sample target category to obtain the first small target loss contribution value includes: Performing a category loss calculation based on the predicted small target category and the sample target category of the adjusted small target to obtain a small target category loss; Performing a category loss calculation based on the predicted large target category and the sample target category of the adjusted large target to obtain a large target category loss; Performing loss calculation on the small object category loss of the training sample set to obtain a total loss of the small object category; Performing loss calculation on the large object category loss of the training sample set to obtain a total loss of the large object category; The first small target loss contribution value is determined based on the total loss of the small target category and the total loss of the large target category.

3. The method according to claim 2, characterized in that The determining the first small target loss contribution value based on the total loss of the small target category and the total loss of the large target category includes: Determine a loss weight based on the first small target area ratio, the loss weight comprising a first loss sub-weight of the total loss of the small target category and a second loss sub-weight of the total loss of the large target category; Perform a weighted loss calculation based on the total loss of the small target category, the first loss sub-weight, the total loss of the large target category, and the second loss sub-weight to obtain a weighted total loss; The loss proportion is calculated based on the total loss of the small target category, the first loss sub-weight and the weighted total loss to obtain the first small target loss contribution value.

4. The method according to claim 1, characterized in that: The step of performing image adjustment on the sample image based on the target size label and the preset target size to obtain an adjustment target and a sample adjusted image containing the adjustment target includes: If the object size labels of the sample objects in the sample image are all larger than the preset object size, determining the maximum object size of the sample image based on the object size labels; determining an image adjustment parameter based on the maximum target size and the preset target size; Determining the number of stitched images based on the image adjustment parameters; Selecting a candidate image from the training sample set based on the number of stitched images; The sample image and the candidate image are subjected to image adjustment based on the image adjustment parameters to obtain the adjustment target and the sample adjusted image including the adjustment target.

5. The method according to claim 4, characterized in that The step of performing image adjustment on the sample image and the candidate image based on the image adjustment parameter to obtain the adjustment target and the sample adjusted image including the adjustment target includes: If the sample image is the same as the candidate image, performing image enhancement on the candidate image to obtain a candidate enhanced image; Performing image size adjustment on the sample image based on the image adjustment parameter to obtain a first adjustment sub-target and a first adjusted image including the first adjustment sub-target; Based on the image adjustment parameter, the candidate enhanced image is resized to obtain a second adjustment sub-target and a second adjusted image including the second adjustment sub-target; Determining the adjustment target based on the first adjustment sub-target and the second adjustment sub-target; Performing image stitching on the first adjustment image and the second adjustment image to obtain a stitched image including the adjustment target, wherein the image size of the stitched image is the same as the image size of the sample image; Perform image smoothing processing on the stitched image to obtain the sample adjusted image.

6. The method according to any one of claims 1 to 5, characterized in that: Before performing image adjustment on the sample image based on the target size label and the preset target size to obtain an adjustment target and a sample adjusted image containing the adjustment target, the method further includes: If the sample image contains the sample target whose target size label is smaller than or equal to the preset target size, the sample target in the sample image is segmented based on the preset target size to obtain a sample small target and a sample large target; Performing target detection on the sample image based on the initial detection model to obtain a second predicted target category of the sample target; Calculate the loss contribution based on the second predicted target category and the sample target category label to obtain a second small target loss contribution value; Determine the area ratio of the second small target based on the total area of ​​the small targets in the sample and the total area of ​​the large targets in the sample in the training sample set; The model parameters of the initial detection model are adjusted based on the second small target loss contribution value and the second small target area ratio to obtain the target detection model.

7. The method according to any one of claims 1 to 5, characterized in that: The adjusting the model parameters of the initial detection model based on the first small target loss contribution value and the first small target area ratio to obtain the target detection model includes: If the first small target loss contribution value is less than the target loss threshold, performing image enhancement on the sample adjusted image to obtain an enhanced adjusted image; updating the training sample set based on the enhanced adjusted image; Determine an updated first small target loss contribution value based on the updated training sample set, and compare the updated first small target loss contribution value with the target loss threshold value until the updated first small target loss contribution value is greater than or equal to the target loss threshold value; The model parameters of the initial detection model are adjusted based on the updated first small target loss contribution value and the first small target area ratio to obtain the target detection model.

8. A target detection device, characterized in that: The device comprises: An acquisition module, configured to acquire a training sample set of a sample target, wherein the training sample set includes a sample image having the sample target and a sample target label of the sample image, wherein the sample target label includes a target size label and a sample target category label of the sample target in the sample image; An adjustment module, configured to perform image adjustment on the sample image based on the target size label and the preset target size, to obtain an adjustment target and a sample adjusted image containing the adjustment target; A division module, used for dividing the adjustment target based on the preset target size to obtain a small adjustment target and a large adjustment target; A first detection module, configured to perform target detection on the sample adjustment image based on a preset initial detection model to obtain a first predicted target category of the adjustment target; A calculation module, configured to calculate a loss contribution based on the first predicted target category and the sample target category label to obtain a first small target loss contribution value; A determination module, configured to determine a first small target area ratio based on a total area of ​​small targets of the adjusted small targets and a total area of ​​large targets of the adjusted large targets in the training sample set; A training module, configured to adjust model parameters of the initial detection model based on the first small target loss contribution value and the first small target area ratio to obtain a target detection model; The second detection module is used to acquire a target image and perform target detection on the target image based on the target detection model.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images