Heterogeneous computing based target detection method, system, and media

Abstract

The application relates to the technical field of image detection, and provides a target detection method and system based on heterogeneous computing and a storage medium. The method comprises the following steps: obtaining, by a first device end, weight data of a target detection model and a to-be-detected image from a host end; loading, by the first device end, the weight data of the target detection model; detecting the to-be-detected image to obtain a target detection frame corresponding to the to-be-detected image; and obtaining a target detection result of the to-be-detected image according to the target detection frame. The weight data of the target detection model is obtained by training a second device end and is stored in the host end. The training of the target detection model is completed by the second device end and the host end, the target detection is completed by the first device end and the host end, the target detection based on a heterogeneous platform and a deep neural network is realized, and the data processing capacity and efficiency of the target detection are improved.

Description

Technical Field

[0001] This application relates to the field of image processing, and in particular to a target detection method, system and medium based on heterogeneous computing. Background Technology

[0002] Since its inception, artificial intelligence (AI) has developed rapidly and has been widely applied in various fields, such as remote sensing imagery, autonomous driving, medical image diagnosis, and natural language processing. Most AI applications are based on image processing technology, primarily divided into image classification, semantic segmentation, and object detection. Lightweight algorithms can accelerate object detection, but with the ever-increasing volume of data, invalid data also increases, and data complexity rises, making it impossible for the data processing speed and computing power of individual computing hardware to meet the demands. Summary of the Invention

[0003] To address the shortcomings of the existing technologies, this invention provides a target detection method, system, and medium based on heterogeneous computing.

[0004] A target detection method based on heterogeneous computing, the method comprising:

[0005] The first device obtains the weight data of the image to be detected and the target detection model from the host device;

[0006] The first device loads the weight data of the target detection model, performs detection on the image to be detected, and obtains the target detection box corresponding to the image to be detected;

[0007] The target detection result of the image to be detected is obtained based on the target detection box;

[0008] The weight data of the target detection model is obtained through training on the second device and stored on the host device.

[0009] In one embodiment of the present invention, before the first device obtains the weight data of the image to be detected and the target detection model from the host, the method further includes:

[0010] The host obtains the training dataset.

[0011] The host sends the training dataset to the second device, which pre-builds a target detection model to be trained and trains the target detection model based on the training dataset to obtain the weight data of the target detection model.

[0012] The second device sends the weight data to the host device.

[0013] In one embodiment of the present invention, the step of obtaining the training dataset on the host side includes:

[0014] The host acquires the initial training set;

[0015] The initial training set is preprocessed to obtain the training dataset. The preprocessing includes at least one of image scaling, image enhancement, and image mean filtering.

[0016] In one embodiment of the present invention, the target detection model to be trained includes: a convolutional layer, a residual block, a pooling layer, and an upsampling layer group; the loss function of the target detection model to be trained includes a classification loss function, a confidence loss function, and an offset loss function.

[0017] In one embodiment of the present invention, the target detection model to be trained is used to obtain a feature map by taking the images in the training dataset as input and passing them through a convolutional layer, a residual block, a pooling layer, and an upsampling group;

[0018] The feature map is subjected to global convolution to obtain the classification result of the target detection model.

[0019] In one embodiment of the present invention, after the second device sends the weight data to the host, the method further includes:

[0020] The host computer performs pruning and quantization operations on the weight data to obtain corrected weight data.

[0021] In one embodiment of the present invention, the first device loads the weight data of the target detection model, performs detection on the image to be detected, and obtains the target detection box corresponding to the image to be detected, including:

[0022] The first device loads the weight data of the target detection model and performs detection on the image to be detected to obtain multiple detection candidate boxes;

[0023] The target candidate box is determined based on the confidence level of any of the detected candidate boxes and the cross-union ratio of the remaining detected candidate boxes.

[0024] A target detection system based on heterogeneous computing, the system comprising a host terminal, a first device terminal, and a second device terminal respectively connected to the host terminal; wherein,

[0025] The host terminal is used to store the weight data of the image to be detected and the target detection model;

[0026] The first device is used to obtain the weight data of the image to be detected and the target detection model from the host, load the weight data of the target detection model, detect the image to be detected, obtain the target detection box corresponding to the image to be detected, and obtain the target detection result of the image to be detected based on the target detection box.

[0027] The second device is used to train the weight data of the target detection model and send it to the host device.

[0028] In one embodiment of the present invention, the host terminal is further configured to: acquire a training dataset; and send it to the second device terminal;

[0029] The second device is further configured to: pre-build a target detection model to be trained, and train the target detection model to be trained according to the training dataset to obtain the weight data of the target detection model.

[0030] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method embodiments.

[0031] This invention acquires the image to be detected and the weight data of the target detection model from the host terminal via a first device. The first device loads the weight data of the target detection model, performs detection on the image to be detected, obtains the target detection box corresponding to the image, and obtains the target detection result of the image based on the target detection box. The weight data of the target detection model is obtained through training on a second device and stored on the host terminal. This invention completes the training of the target detection model through the second device and the host terminal, and performs target detection through the first device and the host terminal, realizing target detection based on heterogeneous platforms and deep neural networks, improving the data processing capability and efficiency of target detection. Attached Figure Description

[0032] Figure 1 This is a logic block diagram of a target detection system based on heterogeneous computing provided in one embodiment of the present invention;

[0033] Figure 2 This is a structural block diagram of a heterogeneous platform provided in one embodiment of the present invention;

[0034] Figure 3 This is a structural block diagram of a heterogeneous platform provided in another embodiment of the present invention;

[0035] Figure 4 This is a logic block diagram of a target detection system based on heterogeneous computing provided in another embodiment of the present invention;

[0036] Figure 5This is a flowchart illustrating a target detection method based on heterogeneous computing provided in one embodiment of the present invention;

[0037] Figure 6 This is a structural block diagram of a target detection algorithm model provided in one embodiment of the present invention;

[0038] Figure 7 This is a structural block diagram of the convolutional layer of the target detection algorithm model provided in one embodiment of the present invention;

[0039] Figure 8 This is a structural block diagram of the residual block of the target detection algorithm model provided in one embodiment of the present invention;

[0040] Figure 9 This is a schematic diagram of the training process of the target detection algorithm model provided in one embodiment of the present invention;

[0041] Figure 10 This is a schematic diagram of the target detection process of the target detection algorithm model provided in one embodiment of the present invention;

[0042] Figure 11 This is an internal structural diagram of a computer device provided in one embodiment of the present invention. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of this application clearer, exemplary embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure may be implemented in various forms and the embodiments not described herein are possible.

[0044] The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The singular forms “a,” “the,” and “the” as used in this disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0045] It should be noted that the terms "first" and "second" used in the embodiments of the present invention are merely used to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first" and "second" can be interchanged in a specific order or sequence where permissible. It should be understood that the objects distinguished by "first" and "second" can be interchanged where appropriate so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein.

[0046] The target detection method based on heterogeneous computing provided in this application can be applied to, for example... Figure 1In the application environment shown, heterogeneous computing platform A is used to perform object detection, and heterogeneous computing platform B is used to train the object detection model. By adopting heterogeneous computing platforms, different task processing functions are assigned to the computing hardware according to its advantages, thus giving full play to the performance of the computing hardware and saving hardware resources.

[0047] like Figure 2 As shown, heterogeneous computing platform A, used for target detection, consists of a host unit and a first device unit. The host unit includes a storage unit and a control unit, possessing powerful serial processing capabilities. It stores data images and weight data of the target detection model, and is responsible for data processing and transmission. The first device unit includes a computing unit, which can acquire data from the host unit and execute the target detection calculation process. The first device unit can be composed of devices with features such as low power consumption, high computing speed, low latency, and hardware programmability, such as FPGAs (Field Programmable Gate Arrays). Software generates hardware combinational logic, adapting to rapid algorithm updates. Furthermore, implementing the algorithm directly in hardware circuitry can compensate for shortcomings such as high latency, insufficient bandwidth, and inadequate parallel computing capabilities, meeting the real-time and continuous requirements of target detection. FPGAs, with their parallel computing capabilities and ultra-low power consumption, can maintain real-time detection.

[0048] like Figure 3 As shown, heterogeneous computing platform B is used for training the object detection model, including a host terminal and a second device terminal. The host terminal provides data and instructions to the device terminal. The second device terminal includes numerous computing units that handle repetitive computational events, such as matrix operations in deep learning. It can perform computational tasks as a device terminal and transmit the weighted files of the computation results back to the host terminal for storage. In some cases, the second device terminal can be a GPU (graphics processing unit).

[0049] In some embodiments, such as Figure 4 As shown, a target detection system based on heterogeneous computing is provided. The system includes a host terminal, a first device terminal, and a second device terminal, which are respectively connected to the host terminal.

[0050] The host computer stores the image to be detected and the weight data of the object detection model. The object detection model is a neural network model, and the weight data is obtained by training this neural network model.

[0051] The first device is used to obtain the weight data of the image to be detected and the target detection model from the host, load the weight data of the target detection model, detect the image to be detected, obtain the target detection box corresponding to the image to be detected, and obtain the target detection result of the image to be detected based on the target detection box.

[0052] The second device is used to train the weight data of the object detection model and send it to the host device.

[0053] The host end is further used to: acquire the training dataset; send it to the second device end; the second device end is further used to: pre-build the target detection model to be trained, and train the target detection model to be trained according to the training dataset to obtain the weight data of the target detection model.

[0054] The target detection system based on heterogeneous computing in the above embodiment completes the training of the target detection model through the second device and the host, and completes the target detection through the first device and the host, thereby realizing target detection based on heterogeneous platforms and deep neural networks, and improving the data processing capability and efficiency of target detection.

[0055] In one embodiment, such as Figure 5 As shown, a target detection method based on heterogeneous computing is provided, which is applied to... Figure 1 A heterogeneous computing platform A performs a target detection process based on heterogeneous computing and neural networks. The method includes the following steps:

[0056] S110, the first device obtains the weight data of the image to be detected and the target detection model from the host.

[0057] The object detection model is a neural network model, trained on the heterogeneous computing platform B based on the training dataset. The weight data of the object detection FPGA model is stored on the host side and can be obtained and loaded by the first device for object detection.

[0058] In the specific implementation, the first device is used to perform the calculation steps of object detection. When performing the object detection steps, the first device can obtain the image to be detected and the weight data of the pre-trained object detection model from the host.

[0059] In some implementations, the host computer can pre-prune and quantize the weight data received from the second device, correcting the weight data to ensure sufficient computational margin for the first device. For example, the host computer can prune the weight file generated by the second device, grouping the four weights into a group and removing the two smallest weights in each group to achieve balanced sparsity without affecting prediction accuracy and speeding up computation. The host computer can also quantize the pruned weight file, using asymmetric quantization to convert floating-point numbers into integers. Asymmetric quantization can better handle unevenly distributed data. By balancing sparsity in the weight file through pruning and quantization to obtain fine-grained sparsity, and then quantizing the data at low bit depths, the number of computational parameters can be effectively reduced.

[0060] S120, the first device loads the weight data of the target detection model, performs detection on the image to be detected, and obtains the target detection box corresponding to the image to be detected;

[0061] In object detection, bounding boxes are typically used to describe the spatial location of an object; these are called detection boxes. In some cases, the first device can identify multiple candidate boxes and select the most suitable one as the object detection box.

[0062] In a specific implementation, the first device can load the weight data of the target detection model and perform target detection on the image to be detected based on the target detection model to obtain the target detection box of the target object.

[0063] For example, the first device can identify multiple candidate detection boxes and determine the target candidate box based on the confidence level of any candidate box and the intersection-union (IU) ratio of the remaining candidate boxes. Specifically, the first device can determine the target candidate box using non-maximum suppression (NMS). This involves calculating the IU ratio of the candidate box with the highest confidence level with the remaining candidates, and deleting the candidate box when the IU ratio exceeds a threshold. This process is repeated until no candidate boxes can be deleted. NMS can effectively delete overlapping candidate boxes of the same category without affecting candidate boxes of other categories.

[0064] S120: Obtain the target detection result of the image to be detected based on the target detection box.

[0065] The object detection result can be either a classification result or a regression result, and is determined based on the detection objective of the pre-trained object detection model.

[0066] In practice, the first device can output the detection result of the image to be detected based on the detection target of the target detection model.

[0067] In the aforementioned object detection method based on heterogeneous computing, a first device acquires the image to be detected and the weight data of the object detection model from the host. The first device loads the weight data of the object detection model, performs detection on the image to be detected, obtains the object detection box corresponding to the image, and obtains the object detection result based on the object detection box. The weight data of the object detection model is trained by a second device and stored on the host. This invention completes the training of the object detection model through the second device and the host, and performs object detection through the first device and the host, realizing object detection based on a heterogeneous platform and deep neural networks, improving the data processing capability and efficiency of object detection.

[0068] In one embodiment, such as Figure 6 The diagram shows the structural block diagram of the object detection model algorithm used in this application. This object detection model uses a fully convolutional residual network method, removing fully connected layers to reduce the number of parameters, and consists of convolutional layers, residual blocks, pooling layers, and upsampling layers.

[0069] refer to Figure 7 Convolutional layers consist of convolution (Conv), normalization (BN), and activation functions (ReLU). Convolution can extract image features, normalization can improve the network's data generalization ability and speed up convergence, and activation functions can improve computational efficiency and maintain data sparsity.

[0070] refer to Figure 8 By stacking residual blocks to extract image features, the model can focus more on features of a certain thickness. Performing max pooling on feature maps of different sizes yields image features of varying thicknesses; after stitching, the model can more effectively detect targets of multiple sizes. The upsampling layer using bilinear interpolation can enlarge the image size while preserving image features. Replacing fully connected layers with global convolutions and 1x1 convolutions effectively reduces the number of parameters and maintains model sparsity.

[0071] In some cases, the loss function can be constructed by adding the classification loss function Loss1, the confidence loss function Loss2, and the offset loss function Loss3. The classification loss function uses MSE (Mean Squared Error) for classification; it determines whether a sample is positive based on the DIOU (Distance Intersection Over Union) value between the candidate box and the ground truth box. A value exceeding a certain threshold is considered positive (1), and below that is considered negative (0). The confidence loss function Loss2 is then constructed using the binary cross-entropy loss function. The offset loss function Loss3 is the mean squared error between the coordinates of the candidate box and the ground truth box. The classification loss function effectively allows the object detection model to adjust weights to output labeled values; the confidence loss function adjusts the center position of the candidate box, making it closer to the ground truth box; and the offset loss function adjusts the size of the box, making the candidate box encompass the target as much as possible.

[0072]

[0073] Where n is the number of samples, y is the label value, and h(x) is the prediction function.

[0074]

[0075]

[0076] Where A is the ground truth bounding box, B is the candidate bounding box, and b and b gt represents the center point of the predicted box and the ground truth box, respectively; p represents the Euclidean distance between the two points; and c represents the farthest distance between the candidate box coordinates and the ground truth box coordinates.

[0077]

[0078] Where DIOU is the intersection-over-union ratio (IoU) between the candidate bounding box and the ground truth bounding box, and gate is a set threshold.

[0079]

[0080] Where n is the number of samples, p is obtained from formula (4), and DIOU is obtained from formula (2).

[0081]

[0082] Where n is the number of samples, and x, y, w, and h are the x-coordinate, y-coordinate, width, and height of the candidate box, respectively. gt y gt w gt h gtThese are the x-coordinate of the center point of the real bounding box, the y-coordinate of the center point, and the width, respectively.

[0083] Loss=Loss1+Loss2+Loss3 Formula (7)

[0084] In one embodiment, the object detection model is... Figure 1 as well as Figure 3 The heterogeneous computing platform B shown completes the training. Figure 9 As shown, the training process of this model includes:

[0085] In step S210, the host acquires images to construct an initial dataset, and divides the dataset into a test set and a training set according to a certain ratio.

[0086] Step S220: The host computer preprocesses the acquired initial dataset to obtain a training dataset for training the object detection model. Preprocessing includes at least one of image scaling, image enhancement, and image mean filtering. The images are resized to a uniform size, image enhancement is used to sharpen them, and mean filtering is used to denoise and optimize the images.

[0087] Scale transformation refers to unifying the image size of the training set, which is beneficial for image processing. Image enhancement: Directly enhancing RGB images can easily alter colors; therefore, RGB images are converted to HSI images, and color enhancement is performed on the I channel to make targets more prominent, which is beneficial for target detection. After image enhancement, the HSI image is converted back to RGB images. Mean filtering: Since image enhancement can easily introduce noise, mean filtering is used to remove noise, making the image smoother and beneficial for image processing.

[0088] Step S230, construct the second device end Figure 6 The target detection model shown, its structure, and the method for constructing the loss function have been described in the above embodiments and will not be repeated here.

[0089] Step S240: Object detection model training. On the second device, images from the training set are used as input to the object detection model to be trained, constructed in step S230, for model training to obtain the weight data of the object detection model. This weight data is then sent to the host for storage. Taking an input image of 512*512*3 as an example, where 512*512 represents the image size and 3 represents the RGB three channels. The first convolutional layer is 3*3*32 / 1, where 3*3 is the kernel size, 32 is the number of output channels, 1 represents the stride, the number of input channels is the same as the number of output channels of the previous layer, and padding is 1. Therefore, after the input image passes through the first convolutional layer, a 512*512*32 feature map is generated. The second convolutional layer is 3*3*64 / 2, where 3*3 is the kernel size, 64 is the number of output channels, 2 represents the stride, the number of input channels is the same as the previous layer's output channels, and padding is 1. Therefore, the feature map after the second convolutional layer is 256*256*64. The feature map then passes through a residual block, where 1X represents one residual block. Since the residual block adds the input and output, stacking residual blocks does not lose image features, and the input and output feature map sizes remain consistent, still 256*256*64. By combining convolutional layers and residual blocks multiple times, a 16*16*1024 feature map is obtained. Stacking different numbers of residual blocks according to the importance of features of varying coarseness allows the model to focus more on features of a particular coarseness. Using the feature map as input to a 1*1*512 / 1 convolutional layer reduces the number of channels and parameters, resulting in a feature map of size 16*16*512. Max pooling layers of 3*3 / 1, 6*6 / 1, and 9*9 / 1 are then applied to the feature map to obtain image features of varying thicknesses. These features are then concatenated with the input, enabling the model to detect targets of multiple sizes. Upsampling is performed on the feature map using bilinear interpolation, enlarging the image size while preserving image features, resulting in a feature map of size 32*32*512. This feature map is then concatenated with the feature map output from an 8X residual block, preventing the loss of important feature information and yielding a feature map of size 32*32*1024. A global convolution is performed on the feature map, with a convolutional layer size of 32*32*1024. Finally, a 1*1*N convolutional layer is used to convolve the feature map, where N = (number of categories + four coordinates of candidate boxes + confidence score) * total number of candidate boxes. This yields the regression and classification results output by the object detection model. By replacing fully connected layers with global convolution and 1*1 convolution, the number of parameters can be effectively reduced, and the sparsity of the model can be maintained.

[0090] Step S250, Object Detection Model Optimization. This step is used to adjust the weight values ​​and optimize the model. Construct as follows... Figure 3 The heterogeneous computing platform B shown uses the CPU as the host to provide data and instructions to the device, and the GPU as the device to complete the computing tasks. Candidate boxes are set; for example, the image can be divided into 16*16 small grids, each with a size of 32*32, and each grid has six candidate boxes with sizes of 16*16, 16*32, 32*16, 48*48, 48*32, and 32*48 respectively. Using different sized candidate boxes can adapt to object detection. The loss function Loss is set according to the formulas (1) to (7) above. The object detection function iteratively updates the weights to minimize the Loss function value. The smaller the Loss value, the closer the predicted candidate box is to the true box. After iteration, a weight file is generated and stored in the CPU.

[0091] The training method for the object detection model in the above embodiments, which completes model training on a heterogeneous computing platform B, facilitates the allocation of different processing tasks based on the resource advantages of the host and device sides, thus saving computing resources. Furthermore, the object detection model uses a fully convolutional residual network, which helps improve the efficiency of feature extraction while maintaining the sparsity of the model.

[0092] In one embodiment, such as Figure 10 As shown, the steps for object detection using a target detection model include:

[0093] Step S310, construct as follows Figure 2 The heterogeneous computing platform A shown in the diagram has the CPU continuing to act as the host, responsible for data processing and data transmission. The FPGA (Field Programmable Gate Array) serves as the first device to perform target detection and output the target detection results.

[0094] Step S320: Weight Pruning and Quantization. To ensure sufficient computational margin for the FPGA, the CPU prunes the weight data generated during the model training phase. This achieves balanced and sparse weights without affecting prediction accuracy, thus accelerating computation. Furthermore, the CPU quantizes the pruned weight data, using asymmetric quantization to convert 32-bit floating-point numbers into 8-bit integers. Asymmetric quantization effectively handles unevenly distributed data. Pruning and quantizing the model weight data effectively reduces the number of computational parameters. It should be understood that this pruning and quantization stage can also be performed on the host machine after step S205.

[0095] Step S330: Perform object detection. The CPU transmits the weight data and image data to the FPGA. The FPGA loads the weight file into the calculation. Each prediction only requires one forward calculation to generate object detection candidate boxes. Therefore, the FPGA's strong parallel computing capability and ultra-low power consumption can maintain real-time detection.

[0096] Step S340: Object detection box filtering. Non-maximum suppression (NMS) is used to filter out the most suitable candidate box. The confidence score of each candidate box is calculated. The Intersection over Union (IOU) between the candidate box with the highest confidence score and the remaining candidates is calculated. When the IOU exceeds a threshold, the candidate box is deleted. This process is repeated until no candidate boxes can be deleted. NMS can effectively delete overlapping candidate boxes of the same category without affecting candidate boxes of other categories.

[0097] Step S350: Output the detection results. After the FPGA completes the target detection task, the target detection results can be directly transmitted to the output terminal, which displays the results. This allows for real-time monitoring of image changes, rapid response, and saves manpower.

[0098] The target detection method in the above embodiments completes the training of the target detection model through the second device and the host, and completes the target detection through the first device and the host, thereby realizing target detection based on heterogeneous platforms and deep neural networks, and improving the data processing capability and efficiency of target detection.

[0099] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the present invention.

[0100] This invention can be deployed on computer devices, such as smartphones, tablets, laptops, desktop computers, rack servers, blade servers, tower servers, or cabinet servers (including standalone servers or server clusters composed of multiple servers). The computer devices in this embodiment include, but are not limited to, memory and processors that can communicate with each other via a system bus. Figure 11 As shown. It should be noted that, Figure 11Only a computer device with memory and processor components is shown; however, it should be understood that it is not required to implement all of the shown components, and more or fewer components may be implemented alternatively. Memory (i.e., readable storage medium) includes flash memory, hard disks, multimedia cards, card-type memories (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disks, optical disks, etc. In some embodiments, memory may be an internal storage unit of the computer device, such as the hard disk or RAM of the computer device. In other embodiments, memory may also be an external storage device of the computer device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. Of course, memory may also include both internal storage units and external storage devices of the computer device. In this embodiment, memory is typically used to store the operating system and various application software installed on the computer device, such as model data, weight data, image data, etc. In addition, the memory can also be used to temporarily store various types of data that have been output or will be output. In some embodiments, the processor can be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. This processor is typically used to control the overall operation of the computer device. In this embodiment, the processor is used to run program code stored in the memory or process data to implement a target detection method based on heterogeneous computing.

[0101] Those skilled in the art will understand that Figure 11 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0102] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0103] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods.

[0104] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A target detection method based on heterogeneous computing, characterized in that, The method includes: The first device obtains the weight data of the image to be detected and the target detection model from the host device; The first device loads the weight data of the target detection model, performs detection on the image to be detected, and obtains the target detection box corresponding to the image to be detected; The target detection result of the image to be detected is obtained based on the target detection box; The weight data of the target detection model is obtained through training on the second device and stored on the host. The host and the first device constitute a heterogeneous computing platform A for performing target detection, and the second device and the host constitute a heterogeneous computing platform B for performing training of the target detection model.

2. The method according to claim 1, characterized in that, Before the first device obtains the weight data of the image to be detected and the target detection model from the host, the method further includes: The host obtains the training dataset. The host sends the training dataset to the second device, which pre-builds a target detection model to be trained and trains the target detection model based on the training dataset to obtain the weight data of the target detection model. The second device sends the weight data to the host device.

3. The method according to claim 2, characterized in that, The steps for obtaining the training dataset on the host side include: The host acquires the initial training set; The initial training set is preprocessed to obtain the training dataset. The preprocessing includes at least one of image scaling, image enhancement, and image mean filtering.

4. The method according to claim 2, characterized in that, The target detection model to be trained includes: convolutional layers, residual blocks, pooling layers, and upsampling layers; the loss function of the target detection model to be trained includes a classification loss function, a confidence loss function, and an offset loss function.

5. The method according to claim 4, characterized in that, The method includes: The target detection model to be trained is used to obtain feature maps by taking images in the training dataset as input and passing them through convolutional layers, residual blocks, pooling layers, and upsampling groups. The feature map is subjected to global convolution to obtain the classification result of the target detection model.

6. The method according to any one of claims 2, characterized in that, After the second device sends the weight data to the host, the method further includes: The host computer performs pruning and quantization operations on the weight data to obtain corrected weight data.

7. The method according to any one of claims 1 to 6, characterized in that, The first device loads the weight data of the target detection model, performs detection on the image to be detected, and obtains the target detection box corresponding to the image to be detected, including: The first device loads the weight data of the target detection model and performs detection on the image to be detected to obtain multiple detection candidate boxes; The target candidate box is determined based on the confidence level of any of the detected candidate boxes and the cross-union ratio of the remaining detected candidate boxes.

8. A target detection system based on heterogeneous computing, characterized in that, The system includes a host terminal, a first device terminal, and a second device terminal, both connected to the host terminal; wherein... The host terminal is used to store the weight data of the image to be detected and the target detection model; The first device is used to obtain the weight data of the image to be detected and the target detection model from the host, load the weight data of the target detection model, detect the image to be detected, obtain the target detection box corresponding to the image to be detected, and obtain the target detection result of the image to be detected based on the target detection box. The second device is used to train the weight data of the target detection model and send it to the host device; wherein the host device and the first device constitute a heterogeneous computing platform A for performing target detection, and the second device and the host device constitute a heterogeneous computing platform B for performing the training of the target detection model.

9. The target detection system based on heterogeneous computing according to claim 8, characterized in that, The host terminal is further used to: acquire the training dataset; and send it to the second device terminal; The second device is further configured to: pre-build a target detection model to be trained, and train the target detection model to be trained according to the training dataset to obtain the weight data of the target detection model.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images