A target detection method, system, device and medium based on an Internet of Things platform

Abstract

The application provides a target detection method, system, device and medium based on an Internet of Things platform, and belongs to the technical field of target detection. The method steps are as follows: a target detection model is improved to obtain a target detection model training network, and the target detection model is trained in combination with collected image data sets; the trained target detection model is exported, optimized and converted into a model file executable by an intelligent camera device; the intelligent camera device loads the model file, processes real-time video streams and obtains target detection results; an Internet of Things display platform displays and monitors the target detection results of the intelligent camera device, and optimizes the target detection results when the target detection results do not meet the requirements. The application separates the target detection model and the training network thereof, obtains effective model features and network parameters, does not increase inference costs, optimizes the trained model and thus efficiently runs in an embedded device.

Description

Technical Field

[0001] This invention belongs to the field of target detection technology, specifically relating to a target detection method, system, device, and medium based on an Internet of Things (IoT) platform. Background Technology

[0002] Target detection technology is a technique that automatically analyzes image or video data to accurately detect the location, size, and category of target objects from input data. With the rapid development of IoT technology, target detection technology is widely used in security monitoring, intelligent transportation, and industrial monitoring. However, existing target detection systems still have shortcomings in real-time performance, detection accuracy, and operating efficiency on embedded devices, especially on resource-constrained embedded devices where target detection efficiency remains low.

[0003] While some deep learning-based object detection algorithms, such as the YOLO series, have made significant progress in accuracy and speed, in practical applications, most deep learning-based object detection models are large in size. The limited hardware performance of embedded devices restricts the direct application of these deep learning models. Furthermore, deep learning models require substantial computational resources for forward inference, especially in complex scenarios involving the protection of multiple objects, where the computational load is enormous and cannot meet the demands of real-time applications. In addition, existing object detection systems also have shortcomings in data transmission, display, and the methods for monitoring and optimizing detection results, failing to provide intuitive and real-time results presentation.

[0004] This is a shortcoming of the existing technology. Therefore, it is very necessary to provide a target detection method, system, device and medium based on the Internet of Things platform to address the above-mentioned defects in the existing technology. Summary of the Invention

[0005] To address the shortcomings of existing target detection methods in practical applications, such as excessively large models, high computational load, and difficulty in efficient operation on embedded devices, this invention provides a target detection method, system, device, and medium based on an Internet of Things (IoT) platform to solve the aforementioned technical problems.

[0006] In a first aspect, the present invention provides a target detection method based on an Internet of Things (IoT) platform, comprising the following steps:

[0007] S1. Improve the original object detection model to obtain the object detection model training network, use the object detection model to train the network, and combine it with the collected image dataset to train the original object detection model;

[0008] S2. Export the trained object detection model and optimize it, then convert the optimized object detection model into a model file that can be executed by smart camera devices;

[0009] S3. The smart camera device loads the model file, processes the real-time video stream using the model file, and obtains the target detection result;

[0010] S4. Display and monitor the target detection results of the smart camera device through the Internet of Things display platform, and optimize the target detection results when they do not meet the requirements.

[0011] Furthermore, the specific steps of step S1 are as follows:

[0012] S11. Obtain the existing target detection model based on the YOLOv7 algorithm;

[0013] S12. Based on the original YOLOv7 algorithm, the multi-branch stacking module is improved by introducing a feature learning component to obtain an improved version of the multi-branch stacking module;

[0014] S13. Construct an auxiliary branch for the original object detection model based on the YOLOv7 algorithm, introduce a feature splitting module and a feature fusion module into the auxiliary branch, and use the feature splitting module and feature fusion module to fuse the improved multi-branch stacking module and convolution operation. Then, superimpose the auxiliary branch with the original object detection model based on the YOLOv7 algorithm to obtain the training network of the object detection model.

[0015] S14. Construct an image dataset using images captured by a smart camera device and object detection images, and divide the image dataset into a training set and a test set;

[0016] S15. Train the original YOLOv7 algorithm's object detection model using the training set through the object detection model's training network, and validate it using the test set after training is complete.

[0017] Furthermore, the specific steps of step S12 are as follows:

[0018] S121. By performing different feature extractions through multiple parallel paths;

[0019] S122. Use feature learning components instead of convolution to learn features;

[0020] S123. Insert aggregation nodes between layers of different depths to fuse the feature maps of different branches, resulting in an improved multi-branch stacked module.

[0021] Furthermore, the specific steps of step S13 are as follows:

[0022] S131. Obtain the input and output of the original target detection model based on the YOLOv7 algorithm, and use them as the input branch and output branch respectively;

[0023] S132. Construct auxiliary input branches using convolution and an improved multi-branch stacking module for the input branches;

[0024] S133. Construct auxiliary output branches for each output branch using convolution and an improved multi-branch stacking module;

[0025] S134. Connect the input of the original target detection model based on the YOLOv7 algorithm to the front end of the input auxiliary branch simultaneously;

[0026] S135. The outputs of the original target detection model based on the YOLOv7 algorithm are split using the feature splitting module;

[0027] S136. The splitting results of each feature splitting module are fused with the back end of the input auxiliary branch and the front end of each output auxiliary branch using the feature fusion module to obtain the training network of the object detection model. The input of the original object detection model based on the YOLOv7 algorithm is the input of the training network of the object detection model, and the output of each output auxiliary branch is the output of the training network of the object detection model.

[0028] The specific steps of step S14 are as follows:

[0029] S141. Use frame extraction to extract data source images from images captured by a smart camera device according to a set ratio;

[0030] S142. Select the corresponding number of target images from the public target detection dataset according to the required targets and quantities;

[0031] S143. Construct an image dataset from the data source images and the target images, and label the data source images as the input and the target images as the output;

[0032] S144. Divide the image dataset into a training set and a test set according to a set ratio.

[0033] Furthermore, the specific steps of step S2 are as follows:

[0034] S21. Export the trained object detection model as an ONXX format;

[0035] S22. Remove weights and nodes with importance below the threshold from the ONXX format object detection model through pruning, convert the model's weights and activation values ​​from floating-point numbers to integers with precision below the threshold, and then merge consecutive computational layers in the object detection model.

[0036] S23. Generate an executable model file for the smart camera device from the optimized target detection model's ONXX format file, and then distribute it to the smart camera device.

[0037] Furthermore, the specific steps of step S3 are as follows:

[0038] S31. Pre-configure the runtime environment and startup script of the executable model file in the smart camera device, wherein the runtime environment includes dependency libraries, drivers and acceleration libraries;

[0039] S32. The smart camera device uses the OpenCV library in the dependency library to acquire real-time video streams and converts them into a format that matches the executable model file;

[0040] S33. After receiving the executable model file, the smart camera device runs the self-starting script and loads the executable model file of the target detection model through the ONNX inference engine;

[0041] S34. The intelligent camera device responds to the user's initialization of the inference session, receives the user's set input and output nodes, calls the inference interface to detect and infer the data source image of the video frame in the real-time video stream, and obtains the inference result;

[0042] S35. The intelligent camera device overlays the inference results onto the real-time video stream to obtain the video stream detection results and displays them in real time.

[0043] Furthermore, the specific steps of step S4 are as follows:

[0044] S41. Establish a transmission channel between the smart camera device and the IoT platform in advance via MQTT or HTTP / HTTPS;

[0045] S42. Configure the network parameters, device ID, and authentication key of the smart camera device, and register it on the IoT platform;

[0046] S43. The smart camera device encapsulates the video stream detection results into messages or data packets, and then sends them to the Internet of Things platform through the transmission channel;

[0047] S44. The IoT platform parses the video stream detection results as messages or data packets and saves them to the database, converts the video stream into image frames, and converts the detection results into JSON format;

[0048] S45. The IoT platform embeds video streams and detection results in the dashboard of the embedded player for users to view;

[0049] S46. The IoT platform monitors the detection results and determines whether the fluctuation between the detection results of consecutive frames exceeds a preset continuous threshold.

[0050] If so, proceed to step S47;

[0051] If not, return to step S3;

[0052] S47. The IoT platform pushes the video stream to the pre-trained target detection model in the background for target detection, and pushes the detection results to the dashboard of the embedded player for display.

[0053] Secondly, the present invention provides a target detection system based on an Internet of Things (IoT) platform, comprising:

[0054] The model building unit is used to improve the original object detection model to obtain the object detection model training network, use the object detection model to train the network, and combine the collected image dataset to train the original object detection model.

[0055] The model conversion unit is used to export the trained object detection model, optimize it, and then convert the optimized object detection model into a model file that can be executed by smart camera devices.

[0056] The video detection unit is used to load model files into the smart camera device, process the real-time video stream using the model files, and obtain target detection results.

[0057] The detection, display, and optimization unit is used to display and monitor the target detection results of smart camera devices through an IoT display platform, and to optimize the target detection results when they do not meet the requirements.

[0058] Thirdly, the present invention provides an electronic device, including a processor and a memory;

[0059] The memory is used to store a computer program, and the processor is used to retrieve and run the computer program from the memory, causing the device to perform the method described in the first aspect above.

[0060] Fourthly, the present invention provides a storage medium,

[0061] The storage medium stores instructions that, when run on a computer, cause the computer to perform the method described in the first aspect above.

[0062] The beneficial effects of this invention are as follows:

[0063] This invention provides a target detection method, system, device, and medium based on an IoT platform. By improving the YOLOv7 algorithm and introducing a multi-branch stacking module and feature learning components, the feature representation capability and computational efficiency of the model are enhanced, enabling efficient real-time processing of target detection on embedded devices and improving detection accuracy. Through optimization techniques such as model pruning, quantization, and layer fusion, the model size and computational load are effectively reduced, significantly improving the model's operating efficiency on resource-constrained embedded devices and ensuring a balance between performance and accuracy. Using the OpenCV library and ONNX Runtime, model inference and video stream processing functions are integrated, enabling real-time target detection of camera video streams and displaying the detection results overlaid on the video stream in real time, providing an intuitive display of detection results. An efficient data transmission channel is established between the smart camera and the IoT platform via MQTT or HTTP / HTTPS protocols, enabling real-time transmission of detection results. The IoT platform displays real-time detection results through an embedded video player and dashboard, improving the user experience. By setting quality indicators for the detection results, the received detection results are monitored in real time. When the detection results are unsatisfactory, a high-precision model on the server is automatically invoked for re-detection to optimize the target detection results, ensuring the stability and reliability of the detection.

[0064] This invention improves the real-time performance, accuracy, and operational efficiency of target detection systems, enhances data transmission and display effects, strengthens the stability and reliability of detection results, and simplifies system integration and management, providing strong support for various practical applications.

[0065] Therefore, it is evident that the present invention has outstanding substantive features and significant progress compared with the prior art, and the beneficial effects of its implementation are also obvious. Attached Figure Description

[0066] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0067] Figure 1 This is a flowchart illustrating the target detection method based on the Internet of Things platform of the present invention.

[0068] Figure 2 This is a schematic diagram of the target detection system based on the Internet of Things platform of the present invention.

[0069] Figure 3 This is a structural framework diagram of the multi-branch stacked module I-ELAN of the present invention.

[0070] Figure 4 This is a structural framework diagram of the feature learning component FLA in the multi-branch stacked module I-ELAN of this invention.

[0071] Figure 5 This is a structural framework diagram of the residual block FLB in the feature learning component FLA of the present invention.

[0072] Figure 6 This is a structural framework diagram of the reparameterized convolution block FLC in the residual block FLB of the present invention.

[0073] Figure 7 This is a structural framework diagram of the target detection model training network with auxiliary branches of the present invention. Detailed Implementation

[0074] The target detection method based on the Internet of Things platform can be applied to one or more devices. The device is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions. Its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0075] The device can be any electronic product that allows human-computer interaction, such as a personal computer, tablet computer, smartphone, personal digital assistant (PDA), or interactive network television (IPTV). The network on which the device operates includes, but is not limited to, the Internet, wide area network (WAN), metropolitan area network (MAN), local area network (LAN), and virtual private network (VPN).

[0076] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0077] Please see Figure 1 The diagram illustrates a target detection method based on an IoT platform in a specific implementation, comprising the following steps:

[0078] S1. Improve the original object detection model to obtain the object detection model training network, use the object detection model to train the network, and combine it with the collected image dataset to train the original object detection model;

[0079] S2. Export the trained object detection model and optimize it, then convert the optimized object detection model into a model file that can be executed by smart camera devices;

[0080] S3. The smart camera device loads the model file, processes the real-time video stream using the model file, and obtains the target detection result;

[0081] S4. Display and monitor the target detection results of the smart camera device through the Internet of Things display platform, and optimize the target detection results when they do not meet the requirements.

[0082] In this embodiment, step S1 is specifically as follows:

[0083] S11. Obtain the existing target detection model based on the YOLOv7 algorithm;

[0084] S12. Based on the original YOLOv7 algorithm, the multi-branch stacking module is improved by introducing a feature learning component to obtain an improved multi-branch stacking module; the specific steps of step S12 are as follows:

[0085] S121. By performing different feature extractions through multiple parallel paths;

[0086] S122. Use feature learning components instead of convolution to learn features;

[0087] S123. Insert aggregation nodes between layers of different depths to fuse feature maps of different branches, resulting in an improved multi-branch stacked module;

[0088] Improved branch stacking module, such as Figure 3 As shown, in the improved branch stacking module I-ELAN, the input feature map is processed through multiple parallel paths for different feature extraction operations, and the feature learning component FLA is used to learn features instead of ordinary convolution; aggregation nodes are inserted between layers of different depths to fuse feature maps from different branch paths. Figure 3 In the diagram, CBS refers to the convolution operation, c1 to c4 refer to the number of channels in each feature layer, and cat refers to the feature layer concatenation operation. The symbols in the following structure diagrams have similar meanings. The improved branch stacking module I-ELAN enhances the feature representation capability of the network through an efficient hierarchical aggregation mechanism while maintaining computational efficiency.

[0089] The structure of FLA is as follows Figure 4As shown, the input feature map is divided into two parts. One part is processed simply by a convolutional layer, and the other part is constructed by stacking N residual blocks (FLBs) to build a deeper network. Finally, the outputs of the two paths are concatenated and output after convolution. Stacking N FLBs can improve the network's ability to express features. Each FLB retains some of the original input features through cross-stage partial connections, while performing more complex feature extraction in the deep path.

[0090] The structure of FLB is as follows: Figure 5 As shown, the input feature map is divided into two parts. One part is processed by convolution through a reparameterized convolutional block (FLC), and the other part is directly passed to the output through a skip connection. The two feature layers are added together by a fusion operation to complete the output. The residual block helps to stabilize the training process, improve the convergence speed, and avoid training difficulties caused by the increase in network depth.

[0091] The structure of FLC is as follows Figure 6 As shown, the input feature map is simultaneously passed to two parallel convolutions. The first convolution has a kernel size of 3×3 and is used to extract local features and details. The second convolution has a kernel size of 1×1 and is used for linear combination and dimensionality reduction between channels. The two feature layers are added together through a fusion operation to complete the output. During the training phase, the FLB structure is a parallel convolution, while during the inference phase, the parallel convolution is reparameterized into a single convolution to speed up inference, reduce computational overhead, optimize the network structure, and improve inference efficiency without affecting model performance.

[0092] The improved branch stacking module I-ELAN uses only traditional convolutions, which have a higher parameter utilization rate than advanced depthwise convolutions, while also having significant advantages in terms of lightweight, speed and accuracy.

[0093] S13. Construct an auxiliary branch for the original YOLOv7-based object detection model. Introduce a feature splitting module and a feature fusion module into the auxiliary branch. Use the feature splitting module and the feature fusion module to fuse the improved multi-branch stacking module and convolution operation. Then, stack the auxiliary branch with the original YOLOv7-based object detection model to obtain the training network of the object detection model. The specific steps of step S13 are as follows:

[0094] S131. Obtain the input and output of the original target detection model based on the YOLOv7 algorithm, and use them as the input branch and output branch respectively;

[0095] S132. Construct auxiliary input branches using convolution and an improved multi-branch stacking module for the input branches;

[0096] S133. Construct auxiliary output branches for each output branch using convolution and an improved multi-branch stacking module;

[0097] S134. Connect the input of the original target detection model based on the YOLOv7 algorithm to the front end of the input auxiliary branch simultaneously;

[0098] S135. The outputs of the original target detection model based on the YOLOv7 algorithm are split using the feature splitting module;

[0099] S136. The splitting results of each feature splitting module are fused with the back end of the input auxiliary branch and the front end of each output auxiliary branch using the feature fusion module to obtain the training network of the object detection model. The input of the original object detection model based on the YOLOv7 algorithm is the input of the training network of the object detection model, and the output of each output auxiliary branch is the output of the training network of the object detection model.

[0100] It should be noted that auxiliary branches are added to the original YOLOv7 algorithm, such as... Figure 7 As shown; OutputA0, OutputB0, and OutputC0 refer to the three feature layers output by the backbone network of the original YOLOv7 algorithm; Inputs refer to the inputs of the backbone network; FLD refers to the feature splitting module, which adjusts the number of channels in its input through convolution and splits it into multiple parts according to the channel dimension; FLE refers to the feature fusion module, which adjusts the multiple parts of its input to the same size and adds the multiple parts to fuse the feature layers; the auxiliary branch processes the feature layers of different scales through feature splitting and feature fusion operations, and outputs three additional feature layers, OutputA1, OutputB1, and OutputC1, which can provide reliable gradient information, obtain more effective target task features, and update network parameters; the auxiliary branch is only used in the training phase, which can retain the inference ability of the original network and does not add additional inference cost.

[0101] S14. Construct an image dataset using images captured by a smart camera device and object detection images, and divide the image dataset into a training set and a test set; the specific steps of step S14 are as follows:

[0102] S141. Use frame extraction to extract data source images from images captured by the smart camera device according to a set ratio; for example, take one image every 10 frames to avoid the similarity of adjacent frame images being too high, which would affect the quality of the dataset and ensure the richness of the dataset.

[0103] S142. Select the corresponding number of target images from the public target detection dataset according to the required targets and quantities;

[0104] S143. Construct an image dataset from the data source images and the target images, and label the data source images as the input and the target images as the output;

[0105] S144. Divide the image dataset into a training set and a test set according to a set ratio;

[0106] S15. Train the original YOLOv7 algorithm's object detection model using the training set through the object detection model's training network, and validate it using the test set after training is complete;

[0107] It should be noted that a training strategy from scratch is used during the training process, and no pre-training weights are set.

[0108] The specific steps of step S2 are as follows:

[0109] S21. Export the trained object detection model as an ONNX format; ONNX supports multiple deep learning frameworks such as TensorFlow, PyTorch, Keras, and Caffe.

[0110] S22. Remove weights and nodes with importance below the threshold from the ONXX format object detection model through pruning, convert the model's weights and activation values ​​from floating-point numbers to integers with precision below the threshold, and then merge consecutive computational layers in the object detection model.

[0111] It should be noted that gradients and activation values ​​are pre-set for the weights or nodes in the target detection model;

[0112] The importance of each weight or node is evaluated based on gradient and activation value metrics, and weights or nodes with importance below a pre-set threshold are removed from the target detection model to complete model pruning.

[0113] The weights and activation values ​​of the target detection model are mapped from floating-point numbers to integers, and the target detection model is calibrated during the quantization process to determine the final quantization parameters and complete the quantization of the target detection model.

[0114] Identify fusionable layers in the object detection model, merge fusionable layers into a single layer, and adjust the weights and biases of the object detection model after fusion;

[0115] The exported ONNX model may contain a lot of redundant information and unoptimized computing nodes. In order to run efficiently on resource-constrained embedded devices, the exported model needs to be optimized through model pruning, quantization and layer fusion. Model pruning removes unimportant weights and nodes to reduce the size and computation of the model. Model quantization reduces the use of storage and computing resources. Merging reduces the storage requirements and computational complexity of intermediate results.

[0116] S23. Generate an executable model file for the smart camera device from the optimized target detection model's ONXX format file, and send it to the smart camera device;

[0117] The specific steps of step S3 are as follows:

[0118] S31. Pre-configure the runtime environment and startup script of the executable model file in the smart camera device, wherein the runtime environment includes dependency libraries, drivers and acceleration libraries;

[0119] S32. The smart camera device uses the OpenCV library in the dependency library to acquire real-time video streams and converts them into a format that matches the executable model file;

[0120] S33. After receiving the executable model file, the smart camera device runs the self-starting script and loads the executable model file of the target detection model through the ONNX inference engine;

[0121] S34. The intelligent camera device responds to the user's initialization of the inference session, receives the user's set input and output nodes, calls the inference interface to detect and infer the data source image of the video frame in the real-time video stream, and obtains the inference result;

[0122] S35. The intelligent camera device overlays the inference results onto the real-time video stream to obtain the video stream detection results and displays them in real time;

[0123] The specific steps of step S4 are as follows:

[0124] S41. Establish a transmission channel between the smart camera device and the IoT platform in advance via MQTT or HTTP / HTTPS;

[0125] S42. Configure the network parameters, device ID, and authentication key of the smart camera device, and register it on the IoT platform;

[0126] S43. The smart camera device encapsulates the video stream detection results into messages or data packets, and then sends them to the Internet of Things platform through the transmission channel;

[0127] S44. The IoT platform parses the video stream detection results as messages or data packets and saves them to the database, converts the video stream into image frames, and converts the detection results into JSON format;

[0128] S45. The IoT platform embeds video streams and detection results in the dashboard of the embedded player for users to view;

[0129] S46. The IoT platform monitors the detection results and determines whether the fluctuation between the detection results of consecutive frames exceeds a preset continuous threshold.

[0130] If so, proceed to step S47;

[0131] If not, return to step S3;

[0132] It should be noted that the continuous threshold can be set according to the actual detection target and scenario; the fluctuation between the detection results of the continuous frames is set by the target type or the number of targets;

[0133] S47. The IoT platform pushes the video stream to the pre-trained target detection model in the background for target detection, and pushes the detection results to the dashboard of the embedded player for display.

[0134] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0135] The following are embodiments of the target detection system based on the Internet of Things (IoT) platform provided in this disclosure. This system belongs to the same inventive concept as the target detection systems based on the IoT platform in the above embodiments. For details not described in detail in the embodiments of the target detection system based on the IoT platform, please refer to the embodiments of the target detection method based on the IoT platform described above.

[0136] like Figure 2 As shown, the system includes:

[0137] The model building unit is used to improve the original object detection model to obtain the object detection model training network, use the object detection model to train the network, and combine the collected image dataset to train the original object detection model.

[0138] The model conversion unit is used to export the trained object detection model, optimize it, and then convert the optimized object detection model into a model file that can be executed by smart camera devices.

[0139] The video detection unit is used to load model files into the smart camera device, process the real-time video stream using the model files, and obtain target detection results.

[0140] The detection, display, and optimization unit is used to display and monitor the target detection results of smart camera devices through an IoT display platform, and to optimize the target detection results when they do not meet the requirements.

[0141] The component comparison method for an energy acquisition device for carbon emission detection provided by this invention comprises modules and algorithm steps of various examples described in conjunction with the embodiments disclosed herein. These modules and steps can be implemented using electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0142] Those skilled in the art will understand that the target detection method for an Internet of Things platform provided by this invention can be implemented as a system, method, or program product. Therefore, the various aspects of this disclosure can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software aspects, collectively referred to herein as a "circuit," "module," or "system."

[0143] The non-transitory computer-readable storage medium for target detection methods stored in an Internet of Things (IoT) platform can be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: electrical connections with one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0144] Although the present invention has been described in detail with reference to the accompanying drawings and preferred embodiments, the present invention is not limited thereto. Various equivalent modifications or substitutions can be made to the embodiments of the present invention by those skilled in the art without departing from the spirit and essence of the invention, and such modifications or substitutions should all be within the scope of the present invention. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should also be covered within the protection scope of the present invention.

Claims

1. A target detection method based on an Internet of Things platform, characterized in that, Comprising the following steps: S1. On the basis of the original target detection model, the target detection model training network is improved, and the original target detection model is trained by using the target detection model training network combined with the collected image data set; The specific steps of step S1 are as follows: S11. Obtain the original target detection model based on YOLOv7 algorithm; S12. On the basis of the original YOLOv7 algorithm, the multi-branch stacking module is improved, and a feature learning component is introduced to obtain an improved multi-branch stacking module; The specific steps of step S12 are as follows: S121. Different feature extraction is performed on multiple parallel paths; S122. Use the feature learning component to replace the convolution to learn the features; The feature learning component divides the input feature map into two parts, one part is simply processed through the convolution layer, and the other part is first processed through the convolution layer, then through the stacking of N residual blocks to construct a deeper network, and finally the outputs of the two paths are spliced and output after convolution processing; S123. Insert aggregation nodes between layers of different depths to fuse feature maps of different branches to obtain an improved multi-branch stacking module; S13. Construct an auxiliary branch for the original target detection model based on YOLOv7 algorithm, introduce a feature splitting module and a feature fusion module in the auxiliary branch, and use the feature splitting module and the feature fusion module to fuse the improved multi-branch stacking module and the convolution operation, then stack the auxiliary branch with the original target detection model based on YOLOv7 algorithm to obtain the target detection model training network; The specific steps of step S13 are as follows: S131. Obtain the input and output of the original target detection model based on YOLOv7 algorithm as the input branch and the output branch respectively; S132. Use convolution and improved multi-branch stacking module to construct input auxiliary branch for input branch; S133. Use convolution and improved multi-branch stacking module to construct output auxiliary branch for each output branch; S134. Connect the input of the original target detection model based on YOLOv7 algorithm to the front end of the input auxiliary branch; S135. Use the feature splitting module to split each output of the original target detection model based on YOLOv7 algorithm; S136. Fuse the splitting results of each feature splitting module with the rear end of the input auxiliary branch and the front end of each output auxiliary branch using the feature fusion module to obtain the target detection model training network, wherein the input of the original target detection model based on YOLOv7 algorithm is the input of the target detection model training network, and the output of each output auxiliary branch is the output of the target detection model training network; S14. Use the images collected by the intelligent camera device and the target detection pictures to construct an image data set, and divide the image data set into a training set and a test set; S15. Train the original YOLOv7 algorithm target detection model using the training set through the target detection model training network, and verify it using the test set after training is completed; S2. Export and optimize the trained target detection model, and convert the optimized target detection model into a model file executable by the intelligent camera device; S3. The intelligent camera device loads the model file, uses the model file to process the real-time video stream, and obtains the target detection result; S4. The target detection result of the intelligent camera device is displayed and monitored through the Internet of Things display platform, and the target detection result is optimized when the target detection result does not meet the requirements. 2.The target detection method based on the Internet of Things platform of claim 1, wherein, Step S14 is specifically as follows: S141. Use the frame extraction method to extract data source pictures from the images collected by the intelligent camera device according to the set proportion; S142. Select a corresponding number of target pictures from the public target detection data set according to the target to be detected and the number; S143. Construct an image data set from the data source pictures and the target pictures, and mark the data source pictures as input parts, and mark the target picture parts as output parts; S144. Divide the image data set into a training set and a test set according to the set proportion. 3.The target detection method based on the Internet of Things platform of claim 1, wherein, Step S2 is specifically as follows: S21. The trained target detection model is exported in ONNX format; S22. Remove weights and nodes with importance lower than a threshold value from the ONNX format target detection model through pruning, and convert the weights and activation values of the model from floating-point numbers to integers with precision lower than a threshold value, and then merge the continuous calculation layers in the target detection model; S23. Generate a model file executable by the intelligent camera device from the ONNX format file of the optimized target detection model, and issue it to the intelligent camera device. 4.The target detection method based on the Internet of Things platform of claim 3, wherein, Step S3 is specifically as follows: S31. Pre-configure the running environment and self-starting script of the executable model file on the intelligent camera device, said running environment including dependent libraries, drivers and acceleration libraries; S32. The intelligent camera device uses the OpenCV library in the dependent library to obtain a real-time video stream, and converts it into a format matching the executable model file; S33. After receiving the executable model file, the intelligent camera device runs the self-starting script, loads the executable model file of the target detection model through the ONNX inference engine; S34. The intelligent camera device responds to the initialization of the inference session by the user, receives user settings for input and output nodes, calls the inference interface to detect the data source pictures in the video frames of the real-time video stream, and obtains the inference result; S35. The intelligent camera device superimposes the inference result on the real-time video stream, obtains the video stream detection result, and displays it in real time. 5.The target detection method based on the Internet of Things platform according to claim 4, wherein, Step S4 is specifically as follows: S41. Pre-establish the transmission channel between the intelligent camera device and the Internet of Things platform through MQTT or HTTP\HTTPS; S42. Configure the network parameters, device ID and authentication key of the intelligent camera device, and register on the Internet of Things platform; S43. The intelligent camera device encapsulates the video stream detection result into a message or data packet, and sends it to the Internet of Things platform through the transmission channel; S44. The Internet of Things platform parses the video stream detection result as a message or data packet and saves it to the database, and converts the video stream into image frames, and converts the detection result into JSON format; S45. The Internet of Things platform embeds the video stream and the detection result in the dashboard of the embedded player and provides it to the user for viewing; S46. The Internet of Things platform monitors the detection result and determines whether the fluctuation between the detection results of consecutive frames exceeds the pre-set continuous threshold; If yes, go to step S47; If no, return to step S3; S47. The Internet of Things platform pushes the video stream to the trained target detection model in the background for target detection, and pushes the detection result to the dashboard of the embedded player for display.

6. An object detection system based on an Internet of Things platform, employing the object detection method based on an Internet of Things platform according to any one of claims 1-5, characterized in that, Comprise: A model construction unit for improving the original target detection model to obtain a target detection model training network, training the original target detection model using the target detection model training network, and combining the collected image data set; A model conversion unit for exporting and optimizing the trained target detection model, and then converting the optimized target detection model into a model file executable by the intelligent camera device; A video detection unit for loading the model file in the intelligent camera device, processing the real-time video stream using the model file, and obtaining the target detection result; A detection display and optimization unit for displaying and monitoring the target detection result of the intelligent camera device through the Internet of Things display platform, and optimizing the target detection result when the target detection result does not meet the requirements.

7. An electronic device, comprising: Comprise a processor and a memory; Wherein the memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that the device executes the method of any one of claims 1-5.

8. A storage medium, characterized in that The storage medium stores instructions that, when executed on a computer, cause the computer to perform the method of any one of claims 1-5.

Images