Target object detection method, device, computer equipment and storage medium

Abstract

The present application relates to a method, device, computer equipment and storage medium for detecting a target object. The method comprises: acquiring a target image; performing target object detection on the target image through a trained target detection model to obtain an initial detection frame and a corresponding confidence level corresponding to each target object in the target image; the target detection model is pre-trained based on a neural network model after dynamically adjusting the network structure; selecting a candidate detection frame from the initial detection frame according to the confidence level; and screening a target detection frame that meets the requirements of the detection scene from the candidate detection frame. The use of this method can improve the detection accuracy and robustness of target object detection.

Description

Technical Field

[0001] The present application relates to the field of target detection technology, and in particular to a target object detection method, device, computer equipment and storage medium. Background Art

[0002] Digitalization and automation are the development direction of all walks of life. In tobacco, postal, pharmaceutical, logistics, chain supermarkets, department stores and manufacturing industries, automated sorting lines are usually set up to detect target objects and perform subsequent digital services based on the detected target objects. Therefore, how to detect target objects on the sorting line is a problem worthy of attention.

[0003] At present, the target object is usually detected from the target image based on image graphics operations such as image blurring, image binarization, opening and closing operations, and edge detection. However, this target object detection method is difficult to adapt to the complex and diverse production environments and target objects of various materials in the detection scene, and has problems such as low detection accuracy and poor robustness. Summary of the invention

[0004] Based on this, it is necessary to provide a target object detection method, device, computer equipment and storage medium that can improve the detection accuracy and robustness of target object detection in response to the above technical problems.

[0005] A target object detection method, the method comprising:

[0006] Get the target image;

[0007] Detecting the target object in the target image by using a trained target detection model to obtain an initial detection frame and a corresponding confidence level corresponding to each target object in the target image; the target detection model is pre-trained based on a neural network model after dynamically adjusting the network structure;

[0008] Selecting a candidate detection frame from the initial detection frame according to the confidence level;

[0009] A target detection frame that meets the detection scenario requirements is selected from the candidate detection frames.

[0010] In one of the embodiments, selecting a target detection frame that meets the detection scene requirement from the candidate detection frames includes:

[0011] Get the pre-configured limiting conditions for the incomplete object detection box;

[0012] The candidate detection frames that match the limited condition are eliminated from the candidate detection frames to obtain the target detection frame.

[0013] In one of the embodiments, selecting a target detection frame that meets the detection scene requirement from the candidate detection frames includes:

[0014] Sort the candidate detection frames in descending order of their sizes to obtain a candidate detection frame sequence;

[0015] Taking the candidate detection frame with the largest detection frame size in the candidate detection frame sequence as a reference detection frame;

[0016] Eliminate, from the candidate detection frame sequence, candidate detection frames whose intersection ratio with the reference detection frame is greater than a preset intersection ratio threshold;

[0017] The candidate detection frames remaining after the elimination process and not including the reference detection frame are used as a new candidate detection frame sequence, and

[0018] Return to the step of taking the candidate detection frame with the largest detection frame size in the candidate detection frame sequence as the reference detection frame and continue to execute until there are less than two candidate detection frames remaining after elimination and excluding the reference detection frame, then stop iteration to obtain the target detection frame.

[0019] In one of the embodiments, removing the candidate detection frames whose intersection ratio with the reference detection frame is greater than a preset intersection ratio threshold from the candidate detection frame sequence includes:

[0020] Calculate the intersection ratio between the reference detection frame and other remaining candidate detection frames;

[0021] Determine, among the combinations of the reference detection frame and other remaining candidate detection frames, a target combination whose corresponding intersection ratio is greater than a preset intersection ratio threshold;

[0022] The candidate detection frame with the smallest detection frame size in each target combination is removed from the candidate detection frame sequence.

[0023] In one embodiment, the step of training the target detection model includes:

[0024] Obtain a training sample set;

[0025] Construct a neural network model after dynamically adjusting the network structure;

[0026] The constructed neural network model is trained based on the training sample set to obtain a trained target detection model.

[0027] In one embodiment, the training of the constructed neural network model based on the training sample set to obtain a trained target detection model includes:

[0028] Training the constructed neural network model based on the training sample set to obtain a trained initial detection model;

[0029] The model parameters of the initial detection model are quantized, and the convolution layer, batch normalization layer and activation layer in the initial detection model are integrated to obtain a trained target detection model.

[0030] In one embodiment, acquiring the target image includes:

[0031] Obtaining an initial image to be detected;

[0032] The initial image is subjected to image preprocessing to obtain a target image.

[0033] A target object detection device, the device comprising:

[0034] An acquisition module, used for acquiring a target image;

[0035] A detection module, used to detect the target object in the target image by using a trained target detection model, and obtain an initial detection frame and a corresponding confidence level corresponding to each target object in the target image; the target detection model is pre-trained based on a neural network model after dynamically adjusting the network structure;

[0036] A selection module, configured to select a candidate detection frame from the initial detection frame according to the confidence level;

[0037] The screening module is used to screen target detection frames that meet the detection scene requirements from the candidate detection frames.

[0038] A computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above-mentioned method embodiments when executing the computer program.

[0039] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps of the above-mentioned method embodiments.

[0040] The above-mentioned target object detection method, device, computer equipment and storage medium can detect the target object of the acquired target image through the trained target detection model, and can quickly and accurately obtain the initial detection frame and confidence corresponding to each target object, select the candidate detection frame from the corresponding initial detection frame based on the confidence with high accuracy, and further screen the target detection frame from the candidate detection frame based on the detection scene requirements, so that based on the target detection frame that meets the detection scene requirements, the target object can be quickly and accurately located from the target image, that is, the detection of the target object can be achieved, and the detection speed and accuracy can be improved. Further, the target detection model is obtained by training the neural network model after dynamically adjusting the network structure, and the target detection model is used to detect the target object of the target image, which can further improve the detection speed and accuracy. In this way, the target detection frame corresponding to the target object in the target image is determined in combination with the target detection model and the detection scene requirements to achieve the detection of the target object, which can be applicable to the detection of target objects of various materials in different production environments under the corresponding detection scenes, has strong robustness, and can improve the detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A diagram of an application environment of a target object detection method in an embodiment;

[0042] Figure 2 is a schematic diagram of a process of a target object detection method in an embodiment;

[0043] Figure 3 A schematic diagram of the principle of a preconfigured incomplete object detection frame in one embodiment;

[0044] Figure 4 A schematic diagram of a box-in-box in a candidate detection box corresponding to a target image in an embodiment;

[0045] Figure 5 A schematic diagram of the principle of optimizing a trained initial detection model to obtain a target detection model in one embodiment;

[0046] Figure 6 is a schematic flow chart of a target object detection method in another embodiment;

[0047] Figure 7 is a schematic diagram of the principle of target object detection in one embodiment;

[0048] Figure 8 is a structural block diagram of a target object detection device in one embodiment;

[0049] Fig. 9 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0051] The target object detection method provided in this application can be applied to Figure 1 In the application environment shown. Wherein, the detection device 102 communicates with the management server 104 through the network. The detection device 102 obtains the target image, performs target object detection on the target image through the trained target detection model, obtains the initial detection frame and the corresponding confidence corresponding to each target object in the target image, selects the candidate detection frame from the initial detection frame according to the confidence, and screens the target detection frame that meets the detection scene requirements from the candidate detection frame, wherein the target detection model is pre-trained based on the neural network model after dynamically adjusting the network structure. The detection device 102 can send the target image and the corresponding target detection frame to the management server 104. Wherein, the detection device 102 can be a terminal or a detection server, and the terminal can be but not limited to various personal computers, laptops, tablet computers and portable wearable devices, and other integrated devices that can be used to realize target object detection, and the integrated device can be specifically integrated with a GPU (Graphics Processing Unit, Graphics Processing Unit). The management server 104 can be implemented with an independent server or a server cluster composed of multiple servers.

[0052] In one embodiment, Figure 2 As shown, a target object detection method is provided, which is applied to Figure 1 The detection equipment in the example is used to illustrate, including the following steps:

[0053] Step 202: Acquire a target image.

[0054] In one embodiment, the detection device receives a target image captured and sent by an image acquisition device. The image acquisition device includes, but is not limited to, an industrial camera and a camera head. The image acquisition device can be built into the detection device as a component, or it can be deployed as an independent device. When the image acquisition device is deployed as an independent device, the image acquisition device can communicate with the detection device via a network, or it can be electrically connected to the detection device. When the image acquisition device communicates with the detection device via a network, the detection device can be deployed in a production environment at the same time as the image acquisition device, so as to reduce the network delay of the image acquisition device sending the target image to the detection device via the network, thereby improving the detection efficiency.

[0055] In one embodiment, the target objects are transmitted and sorted by a sorting line deployed in a production environment, and the corresponding target images are collected by an image acquisition device for the target objects transmitted on the sorting line, so that the detection device can detect the target objects according to the target images. Taking the logistics scenario as an example, the production environment can be a logistics distribution center or a parcel transfer yard, and the parcel transfer yard can specifically be an express delivery transfer yard.

[0056] In one embodiment, the image acquisition device regularly acquires target images according to a preset acquisition cycle, and sends the acquired target images to the detection device, so that the detection device performs corresponding target object detection operations based on the regularly acquired target images. The preset acquisition cycle refers to the time interval between two adjacent acquisitions of target images, such as 2 seconds. It can be understood that the preset acquisition cycle can be specifically consistent with the time interval of the target objects loaded on the sorting line passing through the image acquisition device, so that it can be ensured as much as possible that the image acquisition device acquires a target image for each target object on the sorting line.

[0057] Step 204, target object detection is performed on the target image through the trained target detection model to obtain an initial detection box and a corresponding confidence level corresponding to each target object in the target image; the target detection model is pre-trained based on a neural network model after dynamically adjusting the network structure.

[0058] The target object refers to the object to be detected in the target image. The target objects in different detection scenarios are usually different, such as the target object in the logistics scenario is a package, and the target object in the medical scenario is medical supplies. The initial detection frame can be used to roughly locate the corresponding target object in the target image. Each target object can correspond to one or more initial detection frames. Confidence refers to the degree of credibility that the target object to be detected can be located in the target image based on the corresponding initial detection frame.

[0059] Specifically, in the training phase of the target detection model, a neural network model with dynamically adjusted network structure is constructed, and the constructed neural network model is trained based on the pre-acquired training sample set to obtain a trained target detection model, and the trained target detection model is deployed locally on the detection device. After obtaining the target image to be detected, the detection device performs target object detection on the target image to be detected through the trained target detection model, and obtains the initial detection frame corresponding to each target object in the target image, as well as the confidence corresponding to each initial detection frame.

[0060] In one embodiment, for each target object that can be detected in the target image, the detection device can obtain one or more initial detection frames corresponding to each target object through the trained target detection model. It can be understood that if the detection accuracy of the target detection model is not high enough, the target detection model may mistakenly detect other objects in the target image as the target objects to be detected, and output the corresponding initial detection frames and confidences. Through the post-processing operation provided by the present application, based on the confidence corresponding to each initial detection frame and the detection scene requirements, the initial detection frames corresponding to the target objects with misdetection can be filtered out, so that the target objects to be detected in the target image under the corresponding detection scene can be accurately detected.

[0061] In one embodiment, the neural network model after dynamically adjusting the network structure can be to replace the pooling layer in the network structure of the existing neural network model with a convolutional layer, specifically, to replace the maximum pooling layer in the network structure with a convolutional layer, and to update the kernel size of the maximum pooling layer, and use the updated kernel size as the kernel size of the replaced convolutional layer. In this way, by adjusting the network structure of the neural network model and training the target detection model based on the neural network model after adjusting the network structure, compared with the target detection model obtained by training the neural network model without adjusting the network structure, the detection accuracy can be improved while ensuring efficient detection speed.

[0062] Step 206: Select a candidate detection box from the initial detection box according to the confidence level.

[0063] Among them, the candidate detection box refers to a detection box that may be selected as the target detection box corresponding to the corresponding target object.

[0064] Specifically, the detection device selects a candidate detection frame whose confidence is greater than or equal to a confidence threshold from the initial detection frame corresponding to the target image through a non-maximum suppression post-processing operation based on the initial detection frame corresponding to the target image and the confidence corresponding to each initial detection frame.

[0065] In one embodiment, the step of screening candidate detection frames from the initial detection frame through a non-maximum suppression post-processing operation may specifically include: if the target image corresponds to an initial detection frame, and the confidence corresponding to the initial detection frame is greater than or equal to the confidence threshold, the detection device determines the initial detection frame as a candidate detection frame; if the target image corresponds to multiple initial detection frames, the detection device selects the initial detection frame with the largest confidence from the multiple initial detection frames as the currently selected candidate detection frame, removes the initial detection frames whose overlap with the currently selected candidate detection frame is greater than or equal to the overlap threshold from the multiple initial detection frames, selects the initial detection frame with the largest confidence from the remaining initial detection frames that have not yet been selected as candidate detection frames, and continues to perform the above-mentioned related steps of removing detection frames for the currently selected candidate detection frame until all candidate detection frames are selected.

[0066] Among them, the confidence threshold is used to compare with the confidence of the initial detection frame, so as to select the initial detection frame with a confidence greater than or equal to the confidence threshold from the initial detection frame as the candidate detection frame. It can be understood that when the confidence of the initial detection frame is greater than or equal to the confidence threshold, it indicates that the corresponding target object located from the target image based on the initial detection frame has a high degree of credibility. The overlap threshold is used to compare with the overlap between the initial detection frame and the currently selected candidate detection frame, so as to remove the initial detection frame with a high degree of overlap with the currently selected candidate detection frame from the initial detection frame. It can be understood that when a target object in the target image corresponds to multiple initial detection frames, the overlap between the multiple initial detection frames is relatively high. Therefore, in the above-mentioned manner, the initial detection frame with the largest confidence among the multiple initial detection frames corresponding to the target object is used as the candidate detection frame corresponding to the target object, and other initial detection frames with a high degree of overlap with the initial detection frame are removed. It can be understood that after selecting a candidate detection frame from multiple initial detection frames corresponding to the target image in the above manner, a candidate detection frame with a confidence greater than or equal to a confidence threshold is further selected from the selected candidate detection frame as the candidate detection frame finally selected from the initial detection frame. The confidence threshold and the overlap threshold can be customized according to actual conditions, for example, the confidence threshold is 80% and the overlap threshold is 70%, which are not specifically limited here.

[0067] Step 208: Filter target detection frames that meet the detection scene requirements from the candidate detection frames.

[0068] Among them, the detection scene refers to the scene where the target object to be detected is currently located, such as medical, logistics and manufacturing industries. The detection scene requirements refer to the requirements or conditions that the target object needs to meet in the corresponding detection scene, and specifically can refer to the requirements or conditions that the target detection frame detected for the target object to be detected in the target image needs to meet in the target image. The detection scene requirements are such as the target detection frame corresponding to the target object does not belong to the incomplete object detection frame in the corresponding target image, that is, the limiting conditions corresponding to the pre-configured incomplete object detection frame do not match the target detection frame. The detection scene requirements are also such as each target object corresponds to only one target detection frame in the corresponding target image. It can be understood that if two candidate detection frames are detected for a target object in the target image, and the overlap degree of the two candidate detection frames is relatively high, this situation is called "frame in frame", then the overlap area of ​​the two candidate detection frames covers the same features of the target object, and thus it is necessary to select one of the two candidate detection frames as the target detection frame corresponding to the target object.

[0069] Specifically, after the detection device selects a candidate detection frame from the initial detection frame corresponding to the target image based on the confidence level, it obtains the scene detection requirements preconfigured for the current detection scene, and compares each candidate detection frame with the scene detection requirements to filter the target detection frame corresponding to the corresponding target object from the selected candidate detection frame according to the comparison result, so that the corresponding target object can be located in the corresponding target image based on the target detection frame.

[0070] In one embodiment, target detection frames that meet the detection scene requirements are screened out from the candidate detection frames, so as to extract corresponding target objects from the target image based on the target detection frames, and perform subsequent digitization services based on the extracted target objects.

[0071] In one embodiment, if none of the candidate detection frames corresponding to the target image meets the detection scene requirements, no target detection frame that meets the detection scene requirements can be screened out for the target image, thereby being able to determine that there is no target object in the target image that needs to be detected and meets the detection scene requirements.

[0072] In one embodiment, in order to facilitate the execution of subsequent digitization services based on the detected target object, it is usually required that each target image includes and only includes one target object to be detected, so that a target detection frame can be determined for each target image through the above target object detection method, so as to locate the target object to be detected in the target image based on the target detection frame. If two or more target detection frames are determined for the target image according to the above target object detection method, it may indicate that the target image includes two target objects to be detected, and the target image can be discarded, and corresponding alarm information can be triggered for the target image. If a target detection frame that meets the scene detection requirements is not determined for the target image according to the above target object detection method, it may indicate that the target object to be detected is not placed on the sorting line as required, and the target image can be discarded, and corresponding alarm information can be triggered for the target image.

[0073] The above-mentioned target object detection method performs target object detection on the acquired target image through the trained target detection model, and can quickly and accurately obtain the initial detection frame and confidence corresponding to each target object, select the candidate detection frame from the corresponding initial detection frame based on the confidence with high accuracy, and further screen the target detection frame from the candidate detection frame based on the detection scene requirements, so that based on the target detection frame that meets the detection scene requirements, the target object can be quickly and accurately located from the target image, that is, the detection of the target object can be achieved, and the detection speed and accuracy can be improved. Further, the target detection model is obtained by training the neural network model after dynamically adjusting the network structure, and the target detection model is used to perform target object detection on the target image, which can further improve the detection speed and accuracy. In this way, the target detection frame corresponding to the target object in the target image is determined in combination with the target detection model and the detection scene requirements to achieve the detection of the target object, which can be applicable to the detection of target objects of various materials in different production environments under the corresponding detection scenes, has strong robustness, and can improve the detection accuracy.

[0074] In one embodiment, step 208 includes: obtaining a pre-configured limiting condition for the incomplete object detection frame; and removing the candidate detection frames that match the limiting condition from the candidate detection frames to obtain the target detection frame.

[0075] Among them, the incomplete object detection box refers to the detection box corresponding to the incomplete target object, specifically, it can refer to the detection box detected for the incomplete target object in the target image. The incomplete target object refers to the target object that is incompletely displayed in the target image, rather than the target object itself being incomplete, that is, it means that only a partial area or feature of the target image is captured in the target image. The limiting conditions corresponding to the incomplete object detection box are the conditions or bases for judging whether the candidate detection box matches the corresponding incomplete object detection box, that is, the conditions or bases for judging whether the candidate detection box is an incomplete object detection box.

[0076] Specifically, the detection device pre-configures corresponding incomplete object detection boxes for the target image, and pre-configures corresponding limiting conditions for each incomplete object detection box. During the target object detection process, the detection device obtains the limiting conditions pre-configured for each incomplete object detection box, and compares each candidate detection box corresponding to the target image with the limiting conditions corresponding to each incomplete object detection box respectively. When it is determined that the candidate detection box matches the limiting conditions corresponding to any incomplete object detection box, it is determined that the candidate detection box is an incomplete object detection box, and the candidate detection box determined to be an incomplete object detection box is removed from the candidate detection boxes of the target image object, and then the candidate detection boxes after removing the incomplete object detection boxes are determined as the target detection boxes corresponding to the target image. The detection device can also determine the target detection box according to the comparison results between each candidate object detection box and the limiting conditions, and determine the candidate detection box that does not match all the limiting conditions as the target detection box. It can be understood that during the target object detection process, since the image sizes of each target image are the same, the same incomplete object detection boxes are pre-configured for each target image.

[0077] In one embodiment, there are multiple pre-configured incomplete object detection boxes, and corresponding limiting conditions are pre-configured for each incomplete object detection box respectively. The incomplete object detection box can specifically include an upper incomplete object detection box, a lower incomplete object detection box, a right incomplete object detection box, and a left object detection box, etc. The limiting condition of the upper incomplete object detection box, for example, the maximum ordinate of the candidate detection box is less than the first preset ratio of the height of the target image, that is, y max < h*a, the limiting condition of the lower incomplete object detection box, for example, the minimum ordinate of the candidate detection box is greater than the second preset ratio of the height of the target image, that is, y min > h*(1 - a), the limiting condition of the left incomplete object detection box, for example, the maximum abscissa of the candidate detection box is less than the first preset ratio of the width of the target image, that is, x max < w*a, the limiting condition of the right incomplete object detection box, for example, the minimum abscissa of the candidate detection box is greater than the second preset ratio of the width of the target image, that is, x min > w*(1 - a), where, y max,y min 、x max 、x min Respectively represent the maximum ordinate, minimum ordinate, maximum abscissa and minimum abscissa of the candidate detection frame, h, w, a, (1-a) represent the first preset proportion and the second preset proportion of the target image height and target image width corresponding to the target image. The first preset proportion refers to a preset percentage or percentage, which can be customized according to actual conditions, such as 20%. The sum of the first preset proportion and the second preset proportion is 1.

[0078] Figure 3 FIG. 1 is a schematic diagram of the principle of a preconfigured incomplete object detection frame in an embodiment. Figure 3 As shown, four incomplete object detection frames are preconfigured for the target image, wherein reference numeral 301 represents an upper incomplete object detection frame, reference numeral 302 represents a lower incomplete object detection frame, reference numeral 303 represents a right incomplete object detection frame, and reference numeral 304 represents a left object detection frame. In the target object detection process, based on the above-mentioned limiting conditions, the detection frame data corresponding to each candidate detection frame is respectively compared with the limiting conditions corresponding to each incomplete object detection frame, so as to select the target detection frame according to the comparison result. For example, if the maximum ordinate of the candidate detection frame is less than the first preset proportion of the target image height, that is, if the lower frame of the candidate detection frame is within the upper incomplete object detection frame shown in reference numeral 301, then the candidate detection frame is determined to be the upper incomplete object detection frame. Therefore, if the lower border of the candidate detection frame is in the upper incomplete object detection frame, or the upper border of the candidate detection frame is in the lower incomplete object detection frame, or the left border of the candidate detection frame is in the right incomplete object detection frame, or the right border of the candidate detection frame is in the left incomplete object detection frame, then the candidate detection frame is determined to be an incomplete object detection frame.

[0079] Figure 3 The incomplete object detection frame shown is only used as an example and is not used for specific limitation. It can be understood that since the upper incomplete object detection frame is only used to limit the lower frame of the incomplete object detection frame, Figure 3 The upper incomplete object detection frame shown only needs to have a height consistent with a first preset ratio of the target image height, and there is no specific requirement for its width. For example, the width of the upper incomplete object detection frame can also be consistent with the width of the target image. Figure 3 The reference numeral 305 shown indicates a reference image having the same size as the target image, and is used to illustrate the relationship between each incomplete object detection frame and the target image. Figure 3 The reference numeral 306 shown indicates the location area where the target object is located in the corresponding target image under normal circumstances, that is, indicates the location area where the target detection frame corresponding to the target object is located in the corresponding target image.

[0080] In the above embodiment, based on the limiting conditions corresponding to the incomplete object detection frame, the candidate detection frame belonging to the incomplete object detection frame is eliminated from the candidate detection frame to obtain the target detection frame corresponding to the target object to be detected in the target image.

[0081] In one embodiment, step 208 includes: sorting the candidate detection frames in descending order of detection frame size to obtain a candidate detection frame sequence; taking the candidate detection frame with the largest detection frame size in the candidate detection frame sequence as a reference detection frame; eliminating the candidate detection frames whose intersection ratio with the reference detection frame is greater than a preset intersection ratio threshold from the candidate detection frame sequence; taking the candidate detection frames remaining after the elimination process and not including the reference detection frame as a new candidate detection frame sequence, and returning to the step of taking the candidate detection frame with the largest detection frame size in the candidate detection frame sequence as the reference detection frame to continue executing until the number of candidate detection frames remaining after the elimination process and not including the reference detection frame is less than two, then stopping the iteration to obtain the target detection frame.

[0082] Among them, the candidate detection frame sequence is a detection frame sequence obtained by sorting multiple candidate detection frames in order from large to small according to the detection frame size. The detection frame size refers to the size of the candidate detection frame, which can specifically refer to the product of the height and width of the candidate detection frame. The reference detection frame refers to the candidate detection frame used to compare with other candidate detection frames and determine whether to eliminate other candidate detection frames during the iteration cycle. The intersection ratio refers to the intersection area between the reference detection frame and the candidate detection frame, and the ratio of the minimum area of ​​the reference detection frame and the candidate detection frame to their respective areas, that is, Among them, mIoU represents the intersection ratio between the reference detection frame and the candidate detection frame, |A∩B| represents the intersection area between the reference detection frame A and the candidate detection frame B, and min(A,B) represents the minimum area between the area of ​​the reference detection frame A and the area of ​​the candidate detection frame B. For example, if the area of ​​the reference detection frame A is larger than the area of ​​the candidate detection frame B, then min(A,B) is the area of ​​the reference detection frame A. The preset intersection ratio threshold can be customized according to the actual situation, such as 80%.

[0083] Specifically, if a candidate detection frame is selected from the initial detection frame corresponding to the target image, the detection device determines the candidate detection frame as the target detection frame. If multiple candidate detection frames are selected from the initial detection frame corresponding to the target image, the detection device sorts the multiple candidate detection frames in order of detection frame size from large to small to obtain a corresponding candidate detection frame sequence. The detection device determines the candidate detection frame with the largest detection frame size in the candidate detection frame sequence as the current reference detection frame, calculates the intersection ratio between the reference detection frame and each other candidate detection frame in the candidate detection frame sequence, and according to the calculated intersection ratio, removes the candidate detection frame whose intersection ratio with the reference detection frame is greater than a preset intersection ratio threshold from the candidate detection frame sequence. After the detection device removes the candidate detection frames whose intersection ratio with the reference detection frame is greater than the preset intersection ratio threshold from the candidate detection frame sequence, if the number of candidate detection frames remaining in the candidate detection frame sequence that do not include the reference detection frame is greater than or equal to two, a new candidate detection frame sequence is obtained according to the candidate detection frames remaining in the candidate detection frame sequence that do not include the reference detection frame. For the new candidate detection frame sequence, the detection device takes the candidate detection frame with the largest detection frame size in the candidate detection frame sequence as the current reference detection frame, and returns to the above-mentioned step of calculating the intersection ratio between the reference detection frame and each other candidate detection frame in the candidate detection frame sequence to continue executing until there are less than two candidate detection frames remaining in the candidate detection frame sequence targeted by the current iteration loop and excluding the reference detection frame in the current iteration loop process, then stops the above-mentioned iteration process to obtain the target detection frame corresponding to the target image.

[0084] In one embodiment, if there are two candidate detection frames whose intersection ratios are greater than a preset intersection ratio threshold in the candidate detection frames corresponding to the target image, then the two candidate detection frames can be understood as "frames within frames". Through the above-mentioned iterative elimination processing operation, the candidate detection frames with smaller detection frame sizes can be eliminated from the "frames within frames", and the retained candidate detection frames with larger detection frame sizes can be determined as target detection frames.

[0085] Figure 4 Schematic diagram of a box-in-box in a candidate detection box corresponding to a target image in an embodiment. Figure 4 As shown, reference numeral 401 represents an image with the same image size as the target image, and reference numerals 402 and 403 respectively represent a candidate detection frame corresponding to the target image. Since the intersection ratio between the two candidate detection frames is greater than a preset ratio threshold, the candidate detection frame represented by reference numeral 403 is determined to be a frame-in-frame of the candidate detection frame represented by reference numeral 402. Therefore, the candidate detection frame represented by reference numeral 403 needs to be eliminated, and the candidate detection frame represented by reference numeral 402 needs to be determined as the target detection frame corresponding to the target image.

[0086] In one embodiment, after the detection device selects the candidate detection frame from the initial detection frame corresponding to the target image, the candidate detection frame that matches the limiting conditions corresponding to each pre-configured incomplete object detection frame is removed from the selected candidate detection frame to obtain the candidate detection frame after the incomplete object detection frame is removed, and the in-frame frame is removed from the candidate detection frame after the incomplete object detection frame is removed in accordance with the above-mentioned iterative method to obtain the target detection frame corresponding to the target image. It can be understood that the detection device can also first remove the in-frame frame from the initially acquired candidate detection frame, and then remove the incomplete object detection frame from the candidate detection frame after the in-frame frame is removed.

[0087] In the above embodiment, based on the detection frame size of each candidate detection frame and the intersection ratio between each candidate detection frame, the frame-in-frame is eliminated from the candidate detection frame corresponding to the target image in an iterative manner to obtain the target detection frame corresponding to the target object to be detected in the target image, thereby improving the detection accuracy of the target detection frame.

[0088] In one embodiment, candidate detection frames whose intersection ratio with a reference detection frame is greater than a preset intersection ratio threshold are eliminated from a candidate detection frame sequence, including: calculating the intersection ratio between the reference detection frame and other remaining candidate detection frames; determining, in each combination of the reference detection frame and other remaining candidate detection frames, a target combination whose corresponding intersection ratio is greater than the preset intersection ratio threshold; and eliminating the candidate detection frame with the smallest detection frame size in each target combination from the candidate detection frame sequence.

[0089] Specifically, in each iteration process, after the detection device selects a reference detection frame from the candidate detection frame sequence targeted by the iteration process, it calculates the intersection ratio between the reference detection frame and each other candidate detection frame in the candidate detection frame sequence, and takes the reference detection frame and each other candidate detection frame as a combination to obtain one or more combinations corresponding to the candidate detection frame sequence, and each combination includes the reference detection frame and a candidate detection frame in the candidate detection frame sequence, and takes the intersection ratio between the reference detection frame and the candidate detection frame in each combination as the intersection ratio corresponding to the combination to obtain the intersection ratio corresponding to each combination. Based on the calculated intersection ratio, the detection device determines the combination whose intersection ratio is greater than a preset intersection ratio threshold as the target combination, determines the candidate detection frame with the smallest detection frame size from each target combination, and removes the determined candidate detection frame from the corresponding candidate detection frame sequence.

[0090] In one embodiment, the detection device calculates the intersection ratio between the reference detection frame and each other candidate detection frame in turn, and when the currently calculated intersection ratio is greater than a preset intersection ratio threshold, the candidate detection frame with the smallest detection frame size in the combination corresponding to the intersection ratio may be removed from the candidate detection frame sequence. Alternatively, after calculating the intersection ratio corresponding to each combination, the target combination may be determined based on the calculated intersection ratio, and the candidate detection frame to be deleted may be determined based on the determined target organization, and the determined candidate detection frame may be deleted from the candidate detection frame sequence.

[0091] In the above embodiment, based on the intersection ratio and the preset intersection ratio threshold, the candidate detection frame with a smaller detection frame size is eliminated from the two candidate detection frames whose intersection ratio is greater than the preset intersection ratio threshold, and the candidate detection frame with a larger detection frame size is retained, and the target detection frame is obtained through iterative elimination processing. The target detection frame is the candidate detection frame with a larger detection frame size among the multiple candidate detection frames corresponding to the same target object, so that the corresponding target object can be accurately located from the target image based on the target detection frame.

[0092] In one embodiment, the training steps of the target detection model include: obtaining a training sample set; constructing a neural network model after dynamically adjusting the network structure; and training the constructed neural network model based on the training sample set to obtain a trained target detection model.

[0093] Specifically, the training step of the target detection model is performed by a model training device, and the training step of the target detection model includes: the model training device obtains multiple sample images, manually annotates each target object in each sample image, obtains a sample detection frame corresponding to each sample image, and obtains a training sample set according to the sample image and the corresponding sample detection frame. Accordingly, the model training device dynamically adjusts the network structure of the neural network, and constructs an initial neural network model based on the neural network after adjusting the network structure. The neural network model is a model that has not been trained and cannot be used to detect the corresponding target object from the target image. Further, the model training device trains the constructed neural network model according to the acquired training sample set to obtain a trained target detection model, and the trained target detection model is a model that can be used to quickly and accurately extract the corresponding target object from the target image. The model training device uses the sample image in the training sample set as the input feature, and the corresponding sample detection frame in the training sample set as the desired output feature, and trains the constructed neural network model to obtain a trained target detection model.

[0094] In one embodiment, the model training device may be a detection device for performing target object detection, or a management server, or a model training server specifically used for training models, which is not specifically limited here.

[0095] In one embodiment, a model training device obtains a test sample set, which includes a test image and a test detection frame obtained by manually annotating each test image. After the corresponding target detection model is obtained by training based on the training sample set, the target detection model is tested using the test sample set. If the target detection model is determined to meet preset training requirements based on the test results, the target detection model is determined to be a trained target detection model. Otherwise, the target detection model is continued to be trained based on a new training sample set to obtain a trained target detection model.

[0096] In one embodiment, the sample images in the training sample set and the test images in the test sample set are collected from the sorting line by an image acquisition device. The image acquisition device can send the collected test images and sample images to the data storage device through the industrial computer, so that the model training device can obtain the corresponding sample images and test images from the data storage device. It can be understood that the sample images in the training sample set and the test images in the test sample set can be collected from different production environments under different detection scenarios, so that the target detection model trained based on the training sample set can be applicable to different production environments under different detection scenarios, or can be collected from different production environments under the same detection scenario, so that the target detection model trained based on the training sample set can have high detection accuracy and speed in each production environment under the detection scenario.

[0097] In one embodiment, the neural network model may be constructed based on YOLOv3 after dynamically adjusting the network structure, and specifically may be constructed based on YOLOv3-tiny after dynamically adjusting the network structure. Taking the initial neural network as YOLOv3-tiny as an example, the model training device dynamically adjusts each maxpool layer (maximum pooling layer) with kernel_size (kernel size) = 2*2 and strides (step length) = 2 in the initial neural network to a convolutional layer with kernel_size = 3*3 and strides = 2 (step length), and dynamically adjusts each maxpool layer with kernel_size = 2*2 and strides = 1 in the initial neural network to a convolutional layer with kernel_size = 3*3 and strides = 1, that is, each maximum pooling layer with a kernel size of 2*2 and a step length of 2 in the initial neural network is dynamically adjusted to a convolutional layer with a kernel size of 3*3 and a step length of 2, and each maximum pooling layer with a kernel size of 2*2 and a step length of 1 is dynamically adjusted to a convolutional layer with a kernel size of 3*3 and a step length of 1.

[0098] In one embodiment, the model training device performs image preprocessing on the sample image and the test image, and constructs a training sample set based on the preprocessed sample image, and constructs a test sample set based on the preprocessed test image. Image preprocessing includes but is not limited to image scaling, image padding, and image normalization.

[0099] In one embodiment, during the model training process, the model training device calculates the difference between the predicted detection box output by the neural network model for the sample image and the sample detection box corresponding to the sample image, and dynamically adjusts the model parameters of the neural network model in a direction that reduces the loss function of the neural network model through reverse gradient transfer based on the difference value. The difference between the predicted detection box and the corresponding sample detection box can be determined according to the areas corresponding to the predicted detection box and the sample detection box, respectively, according to a preset mapping relationship. The preset mapping relationship is as follows:

[0100]

[0101]

[0102] Among them, GIoU represents the difference between the predicted detection box A and the sample detection box B, IoU represents the intersection-over-union ratio between the predicted detection box A and the sample detection box B, A∩B represents the intersection of the predicted detection box A and the sample detection box B, and A∪B and U both represent the union of the predicted detection box A and the sample detection box B. c It represents the minimum enclosed area of ​​the predicted detection box A and the sample detection box B, that is, the area of ​​the minimum rectangular box that includes both the predicted detection box A and the sample detection box B.

[0103] It can be understood that existing neural network models usually evaluate the difference value between the predicted detection box and the sample detection box through the intersection and union ratio. The difference value represented by the intersection and union ratio can be used to characterize the distance between the predicted detection box and the sample detection box, but it cannot truly reflect the stacking situation of the target objects in the sample image. The difference value calculated for the predicted detection box and the sample detection box based on the above-mentioned preset mapping relationship can truly reflect the stacking situation of the target objects in the sample image. Therefore, dynamically adjusting the model parameters based on the difference value can improve the accuracy of the model parameters, thereby improving the prediction accuracy of the trained target detection model.

[0104] In one embodiment, during the model training process, after obtaining the sample detection frame corresponding to each sample image through manual annotation, each sample detection frame is clustered into sample detection frames of a preset number of categories according to the corresponding height and width, and the sample detection frame obtained by manual annotation for each sample image is adjusted to the sample detection frame of the corresponding category according to the clustering result, so that when the target object is detected on the target image based on the adjusted sample detection frame and the target detection model trained with the corresponding sample image, the output initial detection frame is any one of the preset number of categories. It can be understood that the initial detection frame output by the target detection model for the template image can be an approximation of any sample detection frame in the above-mentioned preset number of categories. The preset number can be determined according to the number of size types that may exist for the target object in the detection scenario, such as 6.

[0105] In the above embodiment, a neural network model is constructed based on a neural network after dynamically adjusting the network structure, and the constructed neural network model is trained based on a training sample set to obtain a trained target detection model, so that target object detection can be performed on the target image based on the trained target detection model, thereby improving detection accuracy and speed.

[0106] In one embodiment, the constructed neural network model is trained based on the training sample set to obtain a trained target detection model, including: training the constructed neural network model based on the training sample set to obtain a trained initial detection model; quantizing the model parameters of the initial detection model, and integrating the convolution layer, batch normalization layer and activation layer in the initial detection model to obtain the trained target detection model.

[0107] Among them, the initial detection model is a model obtained by training the constructed neural network model based on the training sample set, and the target detection model is a model obtained by optimizing the trained initial detection model.

[0108] Specifically, the model training device trains the constructed neural network model based on the training sample set to obtain a trained initial detection model. The model training device quantizes the model parameters of the trained initial detection model to obtain quantized model parameters, and integrates the trained convolution layer, batch normalization layer and activation layer in the initial detection model to obtain an integrated layer, and then obtains a trained corresponding target detection model based on the quantized model parameters and the integrated integrated layer.

[0109] In one embodiment, the model training device quantizes the model parameters of the initial detection model, which may specifically refer to quantizing the accuracy of the model parameters, such as quantizing the accuracy of the model parameters from FP32 (32-bit floating point) to FP16 (16-bit floating point) or INT8 (8-bit integer). Thus, by quantizing the accuracy of the model parameters, the amount of data of the target detection model can be reduced.

[0110] In one embodiment, the model training device obtains an initial detection model through training on a model training platform, and quantizes the model parameters of the initial detection model through a model optimization platform, and integrates the convolution layer, batch normalization layer and activation layer of the initial detection model to optimize the initial detection model to obtain a corresponding target detection model. Specifically, after the model training device obtains a trained initial detection model based on a training sample set through a model training platform, the initial detection model is converted into a model format that can be recognized by a model optimization platform through the model training platform, and the initial detection model after the format conversion is optimized through the model optimization platform, and the optimized initial detection model is used as a trained target detection model, and the model format of the target detection model is also a model format that can be supported by the model optimization platform. The model training platform includes but is not limited to pytorch (an open source Python machine learning library based on Torch, used for applications such as natural language processing) and TensorFlow (a symbolic mathematical system based on dataflow programming), and the model optimization platform includes but is not limited to TensorRT (a high-performance deep learning inference optimizer).

[0111] Figure 5 The following is a schematic diagram of the principle of optimizing the trained initial detection model to obtain the target detection model in one embodiment. The model training platform is pytorch and TensorFlow, and the model optimization platform is TensorRT. The initial detection model obtained by pytorch training is recorded as pytorch model, and the initial detection model obtained by TensorFlow training is recorded as TensorFlow model. Figure 5As shown in the figure, the process of pytorch model optimization is: converting the model format of pytorch model to obtain ONNX model, optimizing the ONNX model through TensorRT to obtain .trt model, and the .trt model is the target detection model with the model format of .trt; the process of TensorFlow model optimization is: converting the model format of TensorFlow model to obtain .uff model, optimizing the .uff model through TensorRT to obtain .trt model, and the .trt model is the target detection model with the model format of .trt.

[0112] In one embodiment, the target detection model optimized by the model optimization platform can be stored by storing the model parameters and network layers of the target detection model, and the network layer refers to the integration layer obtained by integrating the convolution layer, batch normalization layer and activation layer in the initial detection model. In this way, by obtaining the trained model parameters and the corresponding network layer, and based on the model parameters and the network layer, the trained corresponding target detection model can be created.

[0113] In the above embodiment, the trained initial detection model can be used to quickly and accurately detect the target object from the target image and obtain the initial detection frame corresponding to the target object, but the memory required by the initial detection model is relatively large. By optimizing the initial detection model, a target detection model that requires less memory is obtained, that is, a lightweight target detection model is optimized while ensuring the detection speed and accuracy, so that the target detection model can be applied to various types of detection equipment, such as terminals and management servers deployed in a production environment. It can be understood that the target detection model obtained by model optimization reduces the memory and performance requirements of the detection equipment running the target detection model, thereby reducing equipment costs and improving equipment utilization.

[0114] In one embodiment, step 202 includes: acquiring an initial image to be detected; and performing image preprocessing on the initial image to obtain a target image.

[0115] Specifically, the detection device obtains an initial image to be detected through an image acquisition device according to the image acquisition method provided in one or more of the above embodiments, and performs image preprocessing on the initial image according to a preset preprocessing process to obtain a corresponding target image, so that when the target object is detected on the target image through a trained target detection model, the detection accuracy of the target object can be further improved.

[0116] In one embodiment, image preprocessing includes, but is not limited to, image scaling, image filling, and image normalization. Image scaling refers to scaling the longest side of the initial image to a fixed length in a proportional manner. Image filling refers to filling pixel points with preset pixel values ​​on both sides of the shortest side of the initial image after image scaling to fill the length of the shortest side to a fixed length, so that the longest side of the filled initial image is consistent with the length of the shortest side. Image normalization refers to normalizing the RGB channels of the initial image or the initial image after image filling to 0 to 1 respectively, then subtracting the mean, and finally dividing by the variance to obtain the initial image after image normalization, wherein the detection device can perform image normalization on the initial image or the initial image after image filling according to the following image normalization formula.

[0117] y i =(x i / 255.0-mean i ) / std i

[0118] Among them, i represents the i-th channel in the three RGB channels, mean i Represents the image mean of the i-th channel, std i Represents the image variance of the i-th channel, mean i and std i They are all calculated using sample images in the training sample set during the model training phase.

[0119] It can be understood that the detection device can determine the initial image after image scaling and image filling in sequence as the target image, or can determine the initial image after image scaling, image filling and image normalization in sequence as the target image. In this way, the initial image is adjusted to a target image that meets the input feature requirements of the target detection model through image preprocessing, so that the target object detection model can quickly and accurately detect the target object in the target image.

[0120] In the above embodiment, the target image is obtained by performing image preprocessing on the initial image to be detected, so that when the target object is detected on the target image, the accuracy of the target object detection can be improved.

[0121] like Figure 6 As shown, in one embodiment, a target object detection method is provided, and the method specifically includes the following steps:

[0122] Step 602: Acquire an initial image to be detected.

[0123] Step 604: perform image preprocessing on the initial image to obtain a target image.

[0124] Step 606, target object detection is performed on the target image through the trained target detection model to obtain an initial detection box and a corresponding confidence level corresponding to each target object in the target image; the target detection model is pre-trained based on a neural network model after dynamically adjusting the network structure.

[0125] Step 608: Select a candidate detection box from the initial detection box according to the confidence level.

[0126] Step 610, obtaining the pre-configured limiting conditions for the incomplete object detection frame.

[0127] Step 612: remove the candidate detection frames that match the limited conditions from the candidate detection frames to obtain the target detection frame.

[0128] Step 614, sort the candidate detection frames in descending order of their sizes to obtain a candidate detection frame sequence.

[0129] Step 616: Use the candidate detection frame with the largest detection frame size in the candidate detection frame sequence as a reference detection frame.

[0130] Step 618, calculating the intersection ratio between the reference detection frame and other remaining candidate detection frames.

[0131] Step 620 , among the combinations of the reference detection frame and other remaining candidate detection frames, determine a target combination whose corresponding intersection ratio is greater than a preset intersection ratio threshold.

[0132] Step 622: remove the candidate detection frame with the smallest detection frame size in each target combination from the candidate detection frame sequence.

[0133] Step 624, when the number of candidate detection frames remaining after the elimination process and not including the reference detection frame is greater than or equal to two, the candidate detection frames remaining after the elimination process and not including the reference detection frame are used as new candidate detection frame sequence, and the process returns to step 616 to continue execution.

[0134] Step 626, when the number of candidate detection frames remaining after the elimination process and not including the reference detection frame is less than two, stop falling and obtain the target detection frame corresponding to the target image.

[0135] In the above embodiment, the trained target detection model is obtained based on the training of the neural network model after the dynamic adjustment of the network structure, which can improve the detection speed and accuracy of the target detection model. The target detection model is used to detect the target object of the target image, which can improve the detection speed and accuracy, thereby improving the overall digital operation efficiency. After the initial detection frame and the corresponding confidence corresponding to the target image are predicted by the target detection model, and the candidate detection frame with higher confidence is selected from the initial detection frame based on the confidence, a redundant screening mechanism is adopted to remove the redundant detection frames in the candidate detection frame by removing the incomplete object detection frame and the frame-in-frame from the candidate detection frame, so as to obtain a target detection frame with higher accuracy, which can reduce manual intervention correction and reduce labor costs while improving the detection accuracy. Moreover, this target object detection method can be applied to different production environments, reduce equipment installation and debugging time, and thus reduce labor costs.

[0136] It can be understood that the target object detection method provided in one or more of the above embodiments can be applicable to complex and diverse production environments under detection scenarios. It can ensure detection accuracy while ensuring a high detection speed even when the target image is distorted, deformed, and misaligned due to background light interference, image transmission noise, etc., and when the target object is randomly stacked and placed on the sorting line. This achieves a balance between detection speed and detection accuracy.

[0137] Figure 7 FIG. 1 is a schematic diagram of the principle of target object detection in one embodiment. Figure 7 As shown, the complete target object detection process includes three stages: model training, model optimization and target object detection. Among them, as the model training of stage one, it mainly includes three steps: data set production, network model construction and network training. Among them, data set production mainly refers to obtaining training sample sets, network model construction refers to building a neural network model after dynamically adjusting the network structure, and network training refers to training the constructed neural network model based on the training sample set to obtain a trained initial detection model. The initial detection model obtained in stage one is optimized through stage two to obtain a trained target detection model. It can be understood that stage one and stage two have been executed before stage three, and after the trained target detection model is obtained, there is no need to dynamically execute stage one and stage two in the application stage of the target detection model. Of course, according to actual conditions, the detection accuracy of the target detection model can be further optimized in the application stage.

[0138] Furthermore, the target object detection corresponding to stage three is the application stage of the target detection model. The process of stage three mainly includes: acquiring images, performing image preprocessing on the acquired images, performing model inference on the preprocessed images through the trained target detection model to obtain the corresponding initial detection frame, and filtering out the initial detection frame through the detection frame filtering mechanism to obtain the final detection result.

[0139] In one embodiment, the model training and model optimization operations provided in one or more of the above embodiments may be implemented by a model training device through a GPU (Graphics Processing Unit). Accordingly, during the target object detection process, the target detection model may also be run by the detection device through a GPU.

[0140] It should be understood that although Figure 2 and Figure 6 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 2 and Figure 6 At least part of the steps may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.

[0141] In one embodiment, Figure 8 As shown, a target object detection device 800 is provided, comprising: an acquisition module 801, a detection module 802, a selection module 803 and a screening module 804, wherein:

[0142] An acquisition module 801 is used to acquire a target image;

[0143] The detection module 802 is used to detect the target object in the target image by using the trained target detection model to obtain the initial detection frame and the corresponding confidence level corresponding to each target object in the target image; the target detection model is pre-trained based on the neural network model after dynamically adjusting the network structure;

[0144] A selection module 803 is used to select a candidate detection frame from the initial detection frame according to the confidence level;

[0145] The screening module 804 is used to screen target detection frames that meet the detection scene requirements from the candidate detection frames.

[0146] In one embodiment, the screening module 804 is further used to obtain the pre-configured limiting conditions for the incomplete object detection frame; and remove the candidate detection frames that match the limiting conditions from the candidate detection frames to obtain the target detection frame.

[0147] In one embodiment, the screening module 804 is further used to sort the candidate detection frames in order of detection frame size from large to small to obtain a candidate detection frame sequence; use the candidate detection frame with the largest detection frame size in the candidate detection frame sequence as a reference detection frame; remove the candidate detection frames whose intersection ratio with the reference detection frame is greater than a preset intersection ratio threshold from the candidate detection frame sequence; use the candidate detection frames remaining after the elimination process and not including the reference detection frame as a new candidate detection frame sequence, and return to the candidate detection frame with the largest detection frame size in the candidate detection frame sequence as the reference detection frame to continue execution until the number of candidate detection frames remaining after the elimination process and not including the reference detection frame is less than two, then stop iteration, and obtain the target detection frame.

[0148] In one embodiment, the screening module 804 is also used to calculate the intersection ratio between the reference detection frame and other remaining candidate detection frames; in each combination of the reference detection frame and other remaining candidate detection frames, determine the target combination whose corresponding intersection ratio is greater than a preset intersection ratio threshold; and remove the candidate detection frame with the smallest detection frame size in each target combination from the candidate detection frame sequence.

[0149] In one embodiment, the target object detection device 800 further includes: a model training module;

[0150] The model training module is used to obtain a training sample set; construct a neural network model after dynamically adjusting the network structure; and train the constructed neural network model based on the training sample set to obtain a trained target detection model.

[0151] In one embodiment, the model training module is also used to train the constructed neural network model based on the training sample set to obtain a trained initial detection model; quantize the model parameters of the initial detection model, and integrate the convolution layer, batch normalization layer and activation layer in the initial detection model to obtain a trained target detection model.

[0152] In one embodiment, the acquisition module 801 is further used to acquire an initial image to be detected; and perform image preprocessing on the initial image to obtain a target image.

[0153] For the specific definition of the target object detection device, please refer to the definition of the target object detection method above, which will not be repeated here. Each module in the above-mentioned target object detection device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0154] In one embodiment, a computer device is provided. The computer device may be a terminal as a detection device, and its internal structure diagram may be as follows: Fig. 9 As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a target object detection method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a button, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0155] Those skilled in the art will understand that Fig. 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0156] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps of the above-mentioned method embodiments when executing the computer program.

[0157] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned method embodiments are implemented.

[0158] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0159] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0160] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A target object detection method, characterized in that: The method comprises: Get the target image; Detecting the target object in the target image by using a trained target detection model to obtain an initial detection frame and a corresponding confidence level corresponding to each target object in the target image; the target detection model is pre-trained based on a neural network model after dynamically adjusting the network structure; Selecting a candidate detection frame from the initial detection frame according to the confidence level; Selecting a target detection frame that meets the detection scene requirements from the candidate detection frames; The step of selecting a target detection frame that meets the detection scene requirement from the candidate detection frames includes: Obtaining pre-configured limiting conditions for the incomplete object detection frame; the incomplete object detection frame includes an upper incomplete object detection frame, a lower incomplete object detection frame, a right incomplete object detection frame, and a left incomplete object detection frame; The restriction condition corresponding to the upper incomplete object detection frame is that the maximum vertical coordinate of the candidate detection frame is less than the first preset proportion of the target image height; the restriction condition corresponding to the lower incomplete object detection frame is that the minimum vertical coordinate of the candidate detection frame is greater than the second preset proportion of the target image height; the restriction condition corresponding to the left incomplete object detection frame is that the maximum horizontal coordinate of the candidate detection frame is less than the second preset proportion of the target image height; the restriction condition corresponding to the right incomplete object detection frame is that the minimum horizontal coordinate of the candidate detection frame is greater than the second preset proportion of the target image height; The candidate detection frames that match the limited condition are eliminated from the candidate detection frames to obtain the target detection frame.

2. The method according to claim 1, characterized in that The step of selecting a target detection frame that meets the detection scene requirement from the candidate detection frames includes: Sort the candidate detection frames in descending order of their sizes to obtain a candidate detection frame sequence; Taking the candidate detection frame with the largest detection frame size in the candidate detection frame sequence as a reference detection frame; Eliminate, from the candidate detection frame sequence, candidate detection frames whose intersection ratio with the reference detection frame is greater than a preset intersection ratio threshold; The candidate detection frames remaining after the elimination process and not including the reference detection frame are used as a new candidate detection frame sequence, and Return to the step of taking the candidate detection frame with the largest detection frame size in the candidate detection frame sequence as the reference detection frame and continue to execute until there are less than two candidate detection frames remaining after elimination and excluding the reference detection frame, then stop iteration to obtain the target detection frame.

3. The method according to claim 2, characterized in that The removing, from the candidate detection frame sequence, the candidate detection frames whose intersection ratio with the reference detection frame is greater than a preset intersection ratio threshold, comprises: Calculate the intersection ratio between the reference detection frame and other remaining candidate detection frames; Determine, among the combinations of the reference detection frame and other remaining candidate detection frames, a target combination whose corresponding intersection ratio is greater than a preset intersection ratio threshold; The candidate detection frame with the smallest detection frame size in each target combination is removed from the candidate detection frame sequence.

4. The method according to claim 1, characterized in that: The training steps of the target detection model include: Obtain a training sample set; Construct a neural network model after dynamically adjusting the network structure; The constructed neural network model is trained based on the training sample set to obtain a trained target detection model.

5. The method according to claim 4, characterized in that The step of training the constructed neural network model based on the training sample set to obtain a trained target detection model includes: Training the constructed neural network model based on the training sample set to obtain a trained initial detection model; The model parameters of the initial detection model are quantized, and the convolution layer, batch normalization layer and activation layer in the initial detection model are integrated to obtain a trained target detection model.

6. The method according to any one of claims 1 to 5, characterized in that: The acquiring of the target image comprises: Obtaining an initial image to be detected; The initial image is subjected to image preprocessing to obtain a target image.

7. A target object detection device, characterized in that: The device comprises: An acquisition module, used for acquiring a target image; A detection module, used to detect the target object in the target image by using a trained target detection model, and obtain an initial detection frame and a corresponding confidence level corresponding to each target object in the target image; the target detection model is pre-trained based on a neural network model after dynamically adjusting the network structure; A selection module, configured to select a candidate detection frame from the initial detection frame according to the confidence level; A screening module, used to screen target detection frames that meet the requirements of the detection scene from the candidate detection frames; The screening module is also used to: obtain limiting conditions pre-configured for the incomplete object detection frame; the incomplete object detection frame includes an upper incomplete object detection frame, a lower incomplete object detection frame, a right incomplete object detection frame, and a left incomplete object detection frame; the limiting condition corresponding to the upper incomplete object detection frame is that the maximum vertical coordinate of the candidate detection frame is less than a first preset proportion of the target image height; the limiting condition corresponding to the lower incomplete object detection frame is that the minimum vertical coordinate of the candidate detection frame is greater than a second preset proportion of the target image height; the limiting condition corresponding to the left incomplete object detection frame is that the maximum horizontal coordinate of the candidate detection frame is less than a second preset proportion of the target image height; the limiting condition corresponding to the right incomplete object detection frame is that the minimum horizontal coordinate of the candidate detection frame is greater than the second preset proportion of the target image height; and the candidate detection frames that match the limiting conditions are removed from the candidate detection frames to obtain the target detection frame.

8. The device according to claim 7, characterized in that The screening module is also used to: sort the candidate detection frames in order of detection frame size from large to small to obtain a candidate detection frame sequence; use the candidate detection frame with the largest detection frame size in the candidate detection frame sequence as a reference detection frame; eliminate the candidate detection frames whose intersection ratio with the reference detection frame is greater than a preset intersection ratio threshold from the candidate detection frame sequence; use the candidate detection frames remaining after the elimination process and not including the reference detection frame as a new candidate detection frame sequence, and return to the step of using the candidate detection frame with the largest detection frame size in the candidate detection frame sequence as a reference detection frame to continue executing until the number of candidate detection frames remaining after the elimination process and not including the reference detection frame is less than two, then stop iteration, and obtain a target detection frame.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images