Intelligent object detection method and system based on deep learning, terminal and medium

Abstract

The application relates to an intelligent object detection method and system based on deep learning, a terminal and a medium. The method comprises the following steps: acquiring an image pair comprising a reference image and a target image, determining a correct answer method from a plurality of object detection methods according to a detection time and a detection coincidence rate accuracy, labeling the image pair by using the correct answer method, constructing a training data set comprising the image pair and the correct answer method, inputting the training data set into a pre-constructed deep twin network to obtain a predicted correct answer method, inputting the predicted correct answer method and the correct answer method into a pre-constructed loss function to train the deep twin network, obtaining a trained deep twin network, inputting a to-be-detected image pair into the trained deep twin network to obtain an optimal detection method, detecting the to-be-detected image pair according to the optimal detection method, and obtaining a target object. The method can intelligently select an optimal object detection method for detection in different scenes, and has good universality.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to an intelligent object detection method and system based on deep learning, a terminal and a medium. BACKGROUND

[0002] There are many existing object detection methods, and different methods have different effects in different scenes. For example, the histogram matching method (HM, Histogram Matching) only considers the characteristics of the object as a whole and ignores local details, and is good at solving problems such as object deformation and scaling, but is not good at handling situations with obstacles and changes in illumination. The tone mapping method (TM, Tone Mapping) is good at handling changes in illumination, but has difficulty in handling problems such as deformation and scaling. In addition, some methods have high precision but high computational complexity, which is not needed in some simple use scenarios. Therefore, it is necessary to select the method with the best detection effect for object detection in different scenes. SUMMARY

[0003] Therefore, it is necessary to provide an intelligent object detection method and system based on deep learning that can organically integrate multiple object detection methods into one recognition system to adapt to more scenes.

[0004] An intelligent object detection method based on deep learning, the method comprising:

[0005] An image pair comprising a reference image and a target image is obtained, a random object detection method is selected to perform object detection on the image pair, a detection result and a detection time are obtained, the detection result is compared with a correct answer result of the image pair to obtain a detection result coincidence rate, the detection result coincidence rate and the detection time are calculated to obtain a correct answer method of the image pair, the image pair is labeled according to the correct answer method, and a training data set comprising the image pair and the correct answer method is constructed;

[0006] The training data set is input into a pre-constructed deep twin network for calculation to obtain a predicted correct answer method; wherein the deep twin network comprises a backbone network, a fusion network and a classifier; the attention network in the two groups of backbone networks with the same structure is used to vote and weight the image pair, and reference image features and target image features are extracted; the reference image features and the target image features are fused by the fusion network to obtain final features; and the final features are processed by the classifier to obtain the predicted correct answer method;

[0007] The predicted correct answer method and the correct answer method are input into a pre-constructed loss function for calculation, the deep twin network is trained according to the loss function, and a trained deep twin network is obtained;

[0008] The image pair to be detected is input into the trained deep twin network for calculation to obtain an optimal detection method, and the image pair to be detected is detected according to the optimal detection method to obtain the target object.

[0009] In one embodiment, the detection result is compared with the ground truth of the image pair to obtain a detection result coincidence rate, denoted as

[0010]

[0011] wherein OR i represents the detection result coincidence rate of the i-th object detection method, Ground truth represents the ground truth of the image pair, and Results(i) represents the detection result of the i-th object detection method.

[0012] In one embodiment, the ground truth of the image pair is obtained by calculating the detection result coincidence rate and the detection time, including:

[0013] The coincidence rate threshold and the time threshold are set in advance to calculate the detection result coincidence rate and the detection time to obtain a detection evaluation value, denoted as

[0014]

[0015] wherein score i represents the detection evaluation value of the i-th object detection method, t i represents the detection time of the i-th object detection method, OR th and t th respectively represent the coincidence rate threshold and the time threshold, and λ represents an adjustment parameter.

[0016] The detection evaluation values of multiple object detection methods are compared, and the object detection method with the highest detection evaluation value is selected as the ground truth of the image pair.

[0017] In one embodiment, the attention network in the two groups of structurally identical backbone networks is used to vote and weight the image pair to extract reference image features and target image features, including:

[0018] The target region template is set according to the position region of the target object in the reference image, and the same position in the reference image and the target image is framed according to the target region template;

[0019] The attention network is used to judge the difference of the same pixel point in the target region template of the image pair, and the pixel point is voted according to the difference size to obtain a voting result;

[0020] The voting result is compared with a pre-set voting judgment threshold, and the pixel point is weighted according to the comparison result.

[0021] In one embodiment, the attention network is used to determine the difference of the same pixel point in the target region template of the image pair, the pixel point is voted according to the difference size, and a voting result is obtained, which is represented as

[0022]

[0023]

[0024] wherein k(x, y) represents the voting value, I q represents the reference image, I t represents the target image, pix th represents the pixel point difference threshold, (x, y) represents the pixel point, T represents the target region template, τ = max(mean(I q ), mean(I t )) / min(mean(I q ), mean(I t )) represents the environmental change amount, mean(.) is the average pixel value of the image, max(.) is the maximum value function, and min(.) is the minimum value function.

[0025] In one embodiment, the voting result is compared with a pre-set voting judgment threshold, and a weight is assigned to the pixel point according to the comparison result, including:

[0026] When the voting result of the pixel point is greater than or equal to the voting judgment threshold, the pixel point is determined to be the target object, and a weight greater than 1 is assigned to the pixel point.

[0027] When the voting result of the pixel point is less than the voting judgment threshold, the pixel point is determined to be not the target object, and no weight is assigned to the pixel point.

[0028] In one embodiment, the predicted correct answer method and the correct answer method are input into a pre-constructed loss function to calculate, and the deep twin network is trained according to the loss function to obtain the trained deep twin network, including:

[0029] The pre-constructed loss function is represented as

[0030]

[0031] wherein In(·) represents the indicator function, N is the number of image pairs, represents the predicted correct answer method, o i is the position of the target object in the reference image, GT i represents the correct answer method;

[0032] The prediction ground truth method is input into a loss function for calculation, and the deep twin network is trained according to the loss function, so that the trained deep twin network is obtained.

[0033] An intelligent object detection system based on deep learning, the system comprising:

[0034] A preprocessing module is configured to obtain an image pair comprising a reference image and a target image, randomly select an object detection method to perform object detection on the image pair, obtain a detection result and a detection time, compare the detection result with a ground truth result of the image pair, obtain a coincidence rate of the detection result, calculate according to the coincidence rate of the detection result and the detection time, obtain a ground truth method of the image pair, label the image pair according to the ground truth method, and construct a training data set comprising the image pair and the ground truth method.

[0035] A network construction module is configured to input the training data set into a pre-constructed deep twin network for calculation to obtain a prediction ground truth method; wherein the deep twin network comprises a backbone network, a fusion network and a classifier; the attention networks in the two groups of backbone networks with the same structure are used to vote and weight the image pair, and reference image features and target image features are extracted; the fusion network is used to fuse the reference image features and the target image features to obtain final features; and the classifier is used to process the final features to obtain the prediction ground truth method.

[0036] A training and detection module is configured to input the prediction ground truth method and the ground truth method into a pre-constructed loss function for calculation, train the deep twin network according to the loss function, obtain a trained deep twin network, input a to-be-detected image pair into the trained deep twin network for calculation, obtain an optimal detection method, and detect the to-be-detected image pair according to the optimal detection method to obtain a target object.

[0037] A terminal device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the following steps when executing the computer program:

[0038] An image pair comprising a reference image and a target image is obtained, an object detection method is randomly selected to perform object detection on the image pair, a detection result and a detection time are obtained, the detection result is compared with a ground truth result of the image pair, a coincidence rate of the detection result is obtained, the coincidence rate of the detection result and the detection time are calculated, a ground truth method of the image pair is obtained, the image pair is labeled according to the ground truth method, and a training data set comprising the image pair and the ground truth method is constructed.

[0039] The training data set is input into a pre-constructed deep twin network for calculation to obtain a predicted correct answer method; wherein the deep twin network comprises a backbone network, a fusion network and a classifier; the attention network in the two groups of backbone networks with the same structure votes for the image pair to give power, and reference image features and target image features are extracted; the fusion network fuses the reference image features and the target image features to obtain final features; and the classifier processes the final features to obtain the predicted correct answer method;

[0040] The predicted correct answer method and the correct answer method are input into a pre-constructed loss function for calculation, the deep twin network is trained according to the loss function, and a trained deep twin network is obtained;

[0041] The to-be-detected image pair is input into the trained deep twin network for calculation to obtain an optimal detection method, and the to-be-detected image pair is detected according to the optimal detection method to obtain a target object.

[0042] A terminal readable storage medium has a computer program stored thereon, and the computer program is executed by a processor to implement the following steps:

[0043] An image pair including a reference image and a target image is obtained, an object detection method is randomly selected to detect objects in the image pair to obtain a detection result and a detection time, the detection result is compared with a correct answer result of the image pair to obtain a detection result coincidence rate, the detection result coincidence rate and the detection time are calculated to obtain a correct answer method of the image pair, the image pair is labeled according to the correct answer method, and a training data set including the image pair and the correct answer method is constructed;

[0044] The training data set is input into a pre-constructed deep twin network for calculation to obtain a predicted correct answer method; wherein the deep twin network comprises a backbone network, a fusion network and a classifier; the attention network in the two groups of backbone networks with the same structure votes for the image pair to give power, and reference image features and target image features are extracted; the fusion network fuses the reference image features and the target image features to obtain final features; and the classifier processes the final features to obtain the predicted correct answer method;

[0045] The predicted correct answer method and the correct answer method are input into a pre-constructed loss function for calculation, the deep twin network is trained according to the loss function, and a trained deep twin network is obtained;

[0046] The to-be-detected image pair is input into the trained deep twin network for calculation to obtain an optimal detection method, and the to-be-detected image pair is detected according to the optimal detection method to obtain a target object.

[0047] The intelligent object detection method, system, terminal and medium based on deep learning, by acquiring an image pair including a reference image and a target image, randomly selecting an object detection method to perform object detection on the image pair, comparing the detection result with a correct answer result to obtain a detection coincidence rate, and combining the detection coincidence rate with a detection time, obtains a correct answer method to calibrate the image pair. This calibration method does not need to consider the extraction and distinction of various features, but only uses the performance results of various object detection methods to calibrate data, and combines the time and coincidence rate accuracy during calibration, thereby greatly saving the calculation complexity of data calibration while ensuring the accuracy of the correct answer method. Then, the calibrated image pair and the correct answer method are input into a pre-constructed deep twin network for calculation, and a predicted correct answer method is output. According to the attention network in the two groups of structure-identical backbone networks in the deep twin network, the image pair is voted and weighted to exclude the influence of the background environment of the image pair, so that the network pays more attention to the features of the target object, and higher-precision reference image features and target image features are extracted. Then, the predicted correct answer method and the correct answer method are input into a pre-constructed loss function to train the deep twin network, and a trained deep twin network is obtained. Finally, the image pair to be detected is input into the trained deep twin network for calculation, and an optimal detection method is obtained. According to the optimal detection method, the image pair to be detected is detected to obtain a target object. By using the method, multiple object detection methods can be organically integrated according to the trained deep twin network, and the optimal object detection method can be intelligently selected for detection in different scenes, which has good universality. BRIEF DESCRIPTION OF DRAWINGS

[0048] Figure 1 It is a flowchart of an embodiment of the intelligent object detection method based on deep learning.

[0049] Figure 2 It is an image pair diagram: (a) is a reference image, (b) is a target image.

[0050] Figure 3 It is a structure block diagram of the deep twin network in an embodiment.

[0051] Figure 4 It is a diagram for calculating the detection result coincidence rate in an embodiment.

[0052] Figure 5 It is an image pair framed by a target area template: (a) is a reference image, (b) is a target image.

[0053] Figure 6 It is an internal structure diagram of a computer device in an embodiment. DETAILED DESCRIPTION

[0054] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0055] In one embodiment, such as Figure 1 As shown, a deep learning-based intelligent object detection method is provided, including the following steps:

[0056] Step 102: Obtain image pairs including reference image and target image; randomly select an object detection method to perform object detection on the image pairs; obtain detection results and detection time; compare the detection results with the forward method results of the image pairs to obtain the detection result overlap rate; calculate the forward method of the image pairs based on the detection result overlap rate and detection time; label the image pairs according to the forward method; and construct a training dataset including image pairs and forward method.

[0057] Understandable, such as Figure 2 As shown, the reference image and target image in the image pair refer to images acquired at the same location at different times. The reference image contains... Figure 2 The box in (a) represents the selected target object (Template), in the target image. Figure 2 (b) The boxes represent the detection results of different object detection methods, including the Ground Facts Method (GT), Best Friend Method (BBS), Sum of Squared Differences (SSD), Sum of Absolute Differences (SAD), Normalized Cross-Correlation Method (NCC), BeiDou Navigation Satellite Method (BDS), and Histogram Matching Method (HM). Figure 2 As can be seen, different object detection methods employ different technical means, resulting in significant differences in the detection results.

[0058] It is understandable that a random object detection method is selected to detect objects in the image pair, and the detection results are compared with the forward method results to obtain the detection result overlap rate. The detection overlap rate is then combined with the detection time to obtain the forward method for calibrating the image pair. This calibration method does not need to consider the extraction and differentiation of various features, but only uses the performance results of various object detection methods to calibrate the data. During calibration, the time and overlap rate accuracy are combined, thereby greatly reducing the computational complexity of data calibration while ensuring the accuracy of the forward method.

[0059] Step 104, input the training data set into the pre-constructed deep twin network for calculation to obtain the predicted correct answer method; wherein the deep twin network comprises a backbone network, a fusion network and a classifier; the attention network in the two groups of backbone networks with the same structure is used to vote and weight the image pair to extract the reference image features and the target image features; the fusion network is used to fuse the reference image features and the target image features to obtain the final features; and the classifier is used to process the final features to obtain the predicted correct answer method.

[0060] Specifically, the deep twin network is as shown in the figure. Figure 3 The backbone network refers to two groups of neural networks with the same structure and shared parameters, including an input layer and four convolutional layers. The input of the input layer is a gray image of the reference image and the target image, and the size is 224*224. The first convolutional layer includes an attention network, which is used to vote and weight the image pair to extract target features. The convolution kernel of the first convolutional layer is 5*5, and there are 64 channels. The output is 224*224*64. A first pooling layer is connected after the first convolutional layer, and the maximum pooling kernel is 4*4. The output is 56*56*64. The convolution kernel of the second convolutional layer is 3*3*64, and there are 128 channels. The output is 56*56*128. A second pooling layer is connected after the second convolutional layer, and the maximum pooling kernel is 2*2. The output is 28*28*128. The convolution kernel of the third convolutional layer is 3*3*128, and there are 128 channels. The output is 28*28*128. A third pooling layer is connected after the third convolutional layer, and the maximum pooling kernel is 2*2. The output is 14*14*256. The convolution kernel of the fourth convolutional layer is 3*3*256, and there are 512 channels. The output is 7*7*512. A convolutional layer is connected after the fourth convolutional layer, and the convolution kernel is 7*7*512. The channel is 4096, and the output is 4096.

[0061] The fusion network includes two fully connected layers. The first fully connected layer fuses the reference image features and the target image features output by the backbone network into one feature, and the output is 4096. After going through the second fully connected layer, the output is 4096 to obtain the final features.

[0062] The classifier refers to a softmax layer, which calls various object detection methods to process the final features through the softmax layer to obtain the predicted correct answer method.

[0063] It can be understood that the essence of the deep twin network is to extract target object features. The attention network in the two groups of backbone networks with the same structure in the deep twin network is used to vote and weight the image pair to exclude the influence of the background environment, so that the network pays more attention to the features of the target object, and higher precision reference image features and target image features can be extracted.

[0064] Step 106, the predicted correct answer method and the correct answer method are input into the pre-constructed loss function to calculate, the deep twin network is trained according to the loss function, and the trained deep twin network is obtained.

[0065] Specifically, the pre-constructed loss function is an adaptive loss function.

[0066] Step 108, inputting the image pair to be detected into the trained deep twin network to calculate the optimal detection method, and detecting the image pair to be detected according to the optimal detection method to obtain the target object.

[0067] In the above intelligent object detection method based on deep learning, by acquiring an image pair including a reference image and a target image, randomly selecting an object detection method to detect the image pair, comparing the detection result with the correct answer result to obtain the detection result coincidence rate, and combining the detection coincidence rate with the detection time, the correct answer method is obtained to calibrate the image pair. This calibration method does not need to consider the extraction and distinction of various features, but only uses the performance results of various object detection methods to calibrate data, and combines the time and coincidence rate accuracy during calibration, so as to greatly save the calculation complexity of data calibration under the premise of ensuring the accuracy of the correct answer method. Then, the calibrated image pair and the correct answer method are input into the pre-constructed deep twin network to calculate, the predicted correct answer method is output, and the attention network in the two groups of structure same backbone networks in the deep twin network is voted to weight the image pair, the influence of the background environment in the image pair is excluded, the network pays more attention to the features of the target object, and higher precision reference image features and target image features are extracted. Then, the predicted correct answer method and the correct answer method are input into the pre-constructed loss function to train the deep twin network, and the trained deep twin network is obtained. Finally, the image pair to be detected is input into the trained deep twin network classifier to calculate the optimal detection method, and the image pair to be detected is detected according to the optimal detection method to obtain the target object. By using the method, the various object detection methods of the trained deep twin network can be organically fused, the optimal object detection method can be intelligently selected for detection in different scenes, and good universality is obtained.

[0068] In one embodiment, as shown in Figure 4 the detection result is compared with the correct answer result of the image pair to obtain the detection result coincidence rate, which is represented as

[0069]

[0070] wherein, OR iThe detection result coincidence rate represents the detection result of the ith object detection method, the ground truth represents the correct answer result of the image pair, and the result (i) represents the detection result of the ith object detection method.

[0071] In one embodiment, the correct answer method of the image pair is obtained by calculating the detection result coincidence rate and the detection time.

[0072] The coincidence rate threshold and the time threshold are set in advance to calculate the detection result coincidence rate and the detection time to obtain the detection evaluation value, which is represented as

[0073]

[0074] Wherein, score i The detection evaluation value of the ith object detection method is represented as t i The detection time of the ith object detection method is represented as OR th And t th The coincidence rate threshold and the time threshold are represented as λ, which represents the adjustment parameter.

[0075] The detection evaluation values of multiple object detection methods are compared, and the object detection method with the highest detection evaluation value is selected as the correct answer method of the image pair.

[0076] In one embodiment, the attention network in the two groups of structure same backbone networks is used to vote and weight the image pair to extract the reference image feature and the target image feature, which includes:

[0077] As shown in Figure 5 , first, the target area template is set according to the position area of the target object in the reference image, and the same position in the reference image and the target image is framed according to the target area template, wherein, Figure 5 (a) represents the reference image, and the image box represents the target area template, which marks the target object, Figure 5 (b) represents the target image, and the position of the target area template in the target image is consistent with that in the reference image, but the position of the target object has changed; then the difference of the same pixel point in the target area template of the image pair is judged according to the attention network, the pixel point is voted according to the difference size, and the voting result is obtained; finally, the voting result is compared with the pre-set voting judgment threshold, and the pixel point is weighted according to the comparison result.

[0078] In one embodiment, the difference of the same pixel point in the target area template of the image pair is judged according to the attention network, the pixel point is voted according to the difference size, and the voting result is obtained, which is represented as

[0079]

[0080]

[0081] where k(x, y) represents a voting value, I q represents a reference image, I t represents a target image, pix th represents a pixel point, T represents a target region template, τ = max(mean(I q ), mean(I t )) / min(mean(I q ), mean(I t )) represents an environmental change amount, and the greater the environmental change amount, the looser the determination of the target object, wherein mean(.) is an average pixel value of an image, max(.) is a maximum value function, and min(.) is a minimum value function.

[0082] In one embodiment, the voting result is compared with a pre-set voting judgment threshold, and a weight is assigned to the pixel point according to the comparison result, including:

[0083] When the voting result of the pixel point is greater than or equal to the voting judgment threshold, the pixel point is determined to be a target object, and a weight greater than 1 is assigned to the pixel point.

[0084] When the voting result of the pixel point is less than the voting judgment threshold, the pixel point is determined to be not a target object, and no weight is assigned to the pixel point.

[0085] In one embodiment, the predicted ground truth method and the ground truth method are input into a pre-constructed loss function for calculation, and the deep twin network is trained according to the loss function to obtain the trained deep twin network, including:

[0086] The pre-constructed loss function is represented as

[0087]

[0088] where In(·) represents an indicator function, N is the number of image pairs, represents the predicted ground truth method, o i is the position of the target object in the reference image, and GT i represents the ground truth method.

[0089] The predicted ground truth method and the ground truth method are input into the loss function for calculation, and the deep twin network is trained according to the loss function to obtain the trained deep twin network.

[0090] It should be understood that, although Figure 1The steps in the flowchart of FIG. 1 are displayed in sequence according to the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise explicitly stated herein, there is no strict order limitation for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps in the flowchart of FIG. 1 can include multiple sub-steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily sequential, but can be executed in rotation or alternation with other steps or at least part of the sub-steps or stages of other steps.

[0091] In one embodiment, an intelligent object detection system based on deep learning is provided, comprising a preprocessing module, a network construction module and a training detection module, wherein:

[0092] The preprocessing module is configured to obtain an image pair comprising a reference image and a target image, randomly select an object detection method to perform object detection on the image pair, obtain a detection result and a detection time, compare the detection result with a ground truth result of the image pair to obtain a detection result coincidence rate, calculate according to the detection result coincidence rate and the detection time to obtain a ground truth method of the image pair, label the image pair according to the ground truth method, and construct a training data set comprising the image pair and the ground truth method;

[0093] The network construction module is configured to input the training data set into a pre-constructed deep twin network to calculate a predicted ground truth method; wherein the deep twin network comprises a backbone network, a fusion network and a classifier; the attention network in the two groups of backbone networks with the same structure is used to vote and weight the image pair to extract reference image features and target image features; the fusion network is used to fuse the reference image features and the target image features to obtain final features; and the classifier is used to process the final features to obtain the predicted ground truth method;

[0094] The training detection module is configured to input the predicted ground truth method and the ground truth method into a pre-constructed loss function to calculate, train the deep twin network according to the loss function to obtain a trained deep twin network, input a to-be-detected image pair into the trained deep twin network to calculate an optimal detection method, and detect the to-be-detected image pair according to the optimal detection method to obtain a target object.

[0095] The specific limitations of the intelligent object detection system based on deep learning can refer to the limitations of the intelligent object detection method based on deep learning in the foregoing, which will not be repeated here. Each module in the intelligent object detection system based on deep learning can be realized by software, hardware, and a combination thereof, in whole or in part. The above-mentioned modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so that the processor calls and executes the operations corresponding to each of the above modules.

[0096] In one embodiment, a terminal device is provided, and an internal structure diagram of the terminal device can be as shown in Figure 6 The terminal device includes a processor, a memory, a network interface, a display screen, and an input system connected through a system bus. The processor of the terminal device is configured to provide computing and control capabilities. The memory of the terminal device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The network interface of the terminal device is configured to communicate with an external terminal through a network connection. The computer program is executed by the processor to implement an intelligent object detection method based on deep learning. The display screen of the terminal device can be a liquid crystal display screen or an electronic ink display screen. The input system of the terminal device can be a touch layer overlaid on the display screen, or a key, trackball, or touchpad arranged on the housing of the terminal device, or an external keyboard, touchpad, or mouse, etc.

[0097] Those skilled in the art can understand that Figure 6 The structure shown in the foregoing merely is a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the terminal device to which the scheme of the present application is applied. The specific terminal device can include more or fewer components than those shown in the diagram, or combine certain components, or have a different arrangement of components.

[0098] In one embodiment, a terminal device is provided, including a memory and a processor. The memory stores a computer program. The processor executes the computer program to implement the following steps:

[0099] An image pair including a reference image and a target image is obtained. An object detection method is randomly selected to perform object detection on the image pair, to obtain a detection result and a detection time. The detection result is compared with a correct answer result of the image pair, to obtain a coincidence rate of the detection result. The coincidence rate of the detection result and the detection time are calculated, to obtain a correct answer method of the image pair. The image pair is labeled according to the correct answer method, to construct a training data set including the image pair and the correct answer method.

[0100] The training data set is input into a pre-constructed deep twin network for calculation to obtain a predicted correct answer method; wherein the deep twin network comprises a backbone network, a fusion network and a classifier; the attention network in the two groups of backbone networks with the same structure votes for the image pair to give power, and reference image features and target image features are extracted; the fusion network fuses the reference image features and the target image features to obtain final features; and the classifier processes the final features to obtain the predicted correct answer method;

[0101] The predicted correct answer method and the correct answer method are input into a pre-constructed loss function for calculation, the deep twin network is trained according to the loss function, and a trained deep twin network is obtained;

[0102] The to-be-detected image pair is input into the trained deep twin network for calculation to obtain an optimal detection method, and the to-be-detected image pair is detected according to the optimal detection method to obtain a target object.

[0103] In one embodiment, a terminal readable storage medium is provided, and a computer program is stored on the terminal readable storage medium, and the computer program is executed by a processor to implement the following steps:

[0104] An image pair comprising a reference image and a target image is obtained, an object detection method is randomly selected to detect objects in the image pair to obtain a detection result and a detection time, the detection result is compared with a correct answer result of the image pair to obtain a coincidence rate of the detection result, the coincidence rate of the detection result and the detection time are calculated to obtain a correct answer method of the image pair, the image pair is labeled according to the correct answer method, and a training data set comprising the image pair and the correct answer method is constructed;

[0105] The training data set is input into a pre-constructed deep twin network for calculation to obtain a predicted correct answer method; wherein the deep twin network comprises a backbone network, a fusion network and a classifier; the attention network in the two groups of backbone networks with the same structure votes for the image pair to give power, and reference image features and target image features are extracted; the fusion network fuses the reference image features and the target image features to obtain final features; and the classifier processes the final features to obtain the predicted correct answer method;

[0106] The predicted correct answer method and the correct answer method are input into a pre-constructed loss function for calculation, the deep twin network is trained according to the loss function, and a trained deep twin network is obtained;

[0107] The to-be-detected image pair is input into the trained deep twin network for calculation to obtain an optimal detection method, and the to-be-detected image pair is detected according to the optimal detection method to obtain a target object.

[0108] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, storage, databases, or other media in the embodiments provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0109] The technical features of the above embodiments can be combined in any way. In order to make the description simple, not all possible combinations of the technical features in the above embodiments are described, but as long as the combinations of the technical features do not exist, they should be considered as the scope of the present application.

[0110] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, some modifications and improvements can be made, which are all within the scope of the present application. Therefore, the scope of the patent protection of the present application should be subject to the appended claims.

Claims

1. A deep learning-based intelligent object detection method, characterized in that, The method comprises: obtaining an image pair comprising a reference image and a target image, randomly selecting an object detection method to perform object detection on the image pair to obtain a detection result and a detection time, comparing the detection result with a correct answer result of the image pair to obtain a detection result coincidence rate, calculating according to the detection result coincidence rate and the detection time to obtain a correct answer method of the image pair, marking the image pair according to the correct answer method, and constructing a training data set comprising the image pair and the correct answer method; inputting the training data set into a pre-constructed deep twin network to calculate a predicted correct answer method; wherein the deep twin network comprises a backbone network, a fusion network and a classifier; voting and weighting the image pair according to the attention network in the two groups of structure-identical backbone networks to extract reference image features and target image features; fusing the reference image features and the target image features according to the fusion network to obtain final features; and processing the final features according to the classifier to obtain the predicted correct answer method; inputting the predicted correct answer method and the correct answer method into a pre-constructed loss function to calculate, and training the deep twin network according to the loss function to obtain a trained deep twin network; inputting a to-be-detected image pair into the trained deep twin network to calculate an optimal detection method, and detecting the to-be-detected image pair according to the optimal detection method to obtain a target object; wherein voting and weighting the image pair according to the attention network in the two groups of structure-identical backbone networks to extract reference image features and target image features comprises: setting a target area template according to the position area of the target object in the reference image, and framing the same position in the reference image and the target image according to the target area template; determining the difference of the same pixel point in the target area template of the image pair according to the attention network, voting the pixel point according to the difference size to obtain a voting result; comparing the voting result with a pre-set voting judgment threshold, and assigning a weight to the pixel point according to the comparison result; wherein inputting the predicted correct answer method and the correct answer method into a pre-constructed loss function to calculate, and training the deep twin network according to the loss function to obtain a trained deep twin network comprises: the pre-constructed loss function is represented as wherein, denotes an indicator function, N is the number of image pairs, denotes a prediction oracle method, is the position of the target object in the reference image, denotes an oracle method; inputting the predicted correct answer method and the correct answer method into the loss function to calculate, and training the deep twin network according to the loss function to obtain a trained deep twin network.

2. The method of claim 1, wherein, comparing the detection result with a correct answer result of the image pair to obtain a detection result coincidence rate, represented as wherein, indicates a detection result coincidence rate of a i object detection method, indicates a correct answer result of the image pair, indicates a detection result of a i object detection method.

3. The method of claim 2, wherein, calculating according to the detection result coincidence rate and the detection time to obtain a correct answer method of the image pair, comprising: pre-setting a coincidence rate threshold and a time threshold to calculate the detection result coincidence rate and the detection time to obtain a detection evaluation value, represented as wherein, represents a detection evaluation value of a i object detection method, represents a detection time of a i object detection method, and respectively represent the coincidence rate threshold value and the time threshold value, represents an adjustment parameter; The detection evaluation values of multiple object detection methods are compared, and an object detection method with the highest detection evaluation value is selected as the ground truth method of the image pair.

4. The method of claim 1, wherein, According to the attention network, the difference of the same pixel point in the target region template of the image pair is judged, the pixel point is voted according to the difference size, and a voting result is obtained, denoted as wherein, denotes a voting value, denotes the reference image, denotes the target image, denotes a pixel point difference threshold value, denotes the pixel point, denotes the target region template, denotes an environmental change amount, mean(.) is a function of calculating the average pixel value of an image, max(.) is a function of taking the maximum value, and min(.) is a function of taking the minimum value.

5. The method of claim 4, wherein, The voting result is compared with a pre-set voting judgment threshold, and a weight is assigned to the pixel point according to the comparison result, including: When the voting result of the pixel point is greater than or equal to the voting judgment threshold, the pixel point is determined to be a target object, and a weight greater than 1 is assigned to the pixel point; When the voting result of the pixel point is less than the voting judgment threshold, the pixel point is determined to be not a target object, and no weight is assigned to the pixel point. 6.A deep learning-based intelligent object detection system, characterized in that, The system comprises: A preprocessing module is configured to obtain an image pair comprising a reference image and a target image, randomly select an object detection method to perform object detection on the image pair, obtain a detection result and a detection time, compare the detection result with a ground truth result of the image pair, obtain a detection result coincidence rate, calculate the detection result coincidence rate and the detection time, obtain a ground truth method of the image pair, label the image pair according to the ground truth method, and construct a training data set comprising the image pair and the ground truth method; A network construction module is configured to input the training data set into a pre-constructed deep twin network to obtain a predicted ground truth method; wherein the deep twin network comprises a backbone network, a fusion network and a classifier; the attention network in the two groups of backbone networks with the same structure is used to vote and weight the image pair, and reference image features and target image features are extracted; the reference image features and the target image features are fused by the fusion network to obtain final features; and the final features are processed by the classifier to obtain the predicted ground truth method; A training detection module is configured to input the predicted ground truth method and the ground truth method into a pre-constructed loss function to calculate the loss function, train the deep twin network according to the loss function, obtain a trained deep twin network, input a to-be-detected image pair into the trained deep twin network to calculate an optimal detection method, and detect the to-be-detected image pair according to the optimal detection method to obtain a target object. According to the attention network, the difference of the same pixel point in the target region template of the image pair is judged, the pixel point is voted according to the difference size, and a voting result is obtained, denoted as The voting result is compared with a pre-set voting judgment threshold, and a weight is assigned to the pixel point according to the comparison result, including: When the voting result of the pixel point is greater than or equal to the voting judgment threshold, the pixel point is determined to be a target object, and a weight greater than 1 is assigned to the pixel point; When the voting result of the pixel point is less than the voting judgment threshold, the pixel point is determined to be not a target object, and no weight is assigned to the pixel point. The pre-constructed loss function is calculated by inputting the predicted positive solution method and the positive solution method into the loss function, and the deep twin network is trained according to the loss function to obtain a trained deep twin network, comprising: The pre-constructed loss function is expressed as wherein, represents an indicator function, N is the number of image pairs, represents a prediction oracle method, is the position of the target object in the reference image, represents an oracle method; The pre-constructed loss function is calculated by inputting the predicted positive solution method and the positive solution method into the loss function, and the deep twin network is trained according to the loss function to obtain a trained deep twin network. 7.A terminal device, comprising a memory and a processor, wherein the memory stores a computer program, and the terminal device is characterized in that, The processor executes the computer program to realize the steps of the method in any one of claims 1 to 5.

8. A terminal-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to realize the steps of the method in any one of claims 1 to 5.

Images