Target detection method, system and device based on Internet of Things platform and medium

Abstract

The invention provides a target detection method, system and device based on an Internet of Things platform and a medium, and belongs to the technical field of target detection, and the method comprises the following steps: improving a target detection model to obtain a target detection model training network, and training the target detection model in combination with a collected image data set; exporting the trained target detection model, optimizing the model, and converting the model into a model file which can be executed by the intelligent camera device; the intelligent camera device loads the model file and processes the real-time video stream to obtain a target detection result; and the Internet of Things display platform displays and monitors the target detection result of the intelligent camera device, and optimizes the target detection result when the target detection result does not meet the requirement. According to the method, the target detection model and the training network are separated, effective model features and network parameters are obtained, the reasoning cost is not increased, and the trained model is optimized, so that efficient operation in the embedded equipment is realized.

Description

Background Art

[0002] Object detection technology is a technology that automatically analyzes image or video data to accurately detect the location, size, and category of the target object from the input data. With the rapid development of Internet of Things technology, object detection technology is widely used in security monitoring, intelligent transportation, industrial monitoring and other fields. Existing object detection systems still have deficiencies in real-time performance, detection accuracy, and operating efficiency on embedded devices. In particular, on resource-constrained embedded devices, the object detection efficiency is still low.

[0003] Although some deep learning target detection algorithms, such as the YOLO series, have made significant progress in accuracy and speed, in practical applications, most deep learning-based target detection models are large in size, and the hardware performance of embedded devices is limited, which limits the direct application of these deep learning models. In addition, deep learning models require a lot of computing resources when performing forward reasoning, especially for complex scenarios where target detection personnel protect multiple objects. The amount of calculation is huge and cannot meet the needs of real-time applications. In addition, the existing target detection system also has deficiencies in data transmission, display, and actual monitoring and optimization methods of detection results, and cannot display results intuitively and in real time.

[0004] This is a shortcoming of the prior art. Therefore, in view of the above-mentioned defects in the prior art, it is very necessary to provide a target detection method, system, device and medium based on the Internet of Things platform. Summary of the invention

[0005] In view of the fact that the above-mentioned existing target detection methods still face the defects of being too large in model, too much calculation and difficult to run efficiently on embedded devices in practical applications, the present invention provides a target detection method, system, device and medium based on the Internet of Things platform to solve the above-mentioned technical problems.

[0006] In a first aspect, the present invention provides a target detection method based on an Internet of Things platform, comprising the following steps: S1. Improve the original target detection model to obtain a target detection model training network, use the target detection model training network, and train the original target detection model in combination with the collected image data set; S2. Export and optimize the trained target detection model, and then convert the optimized target detection model into a model file executable by the smart camera device; S3. The smart camera device loads the model file and uses the model file to process the real-time video stream to obtain the target detection result; S4. Display and monitor the target detection results of the smart camera device through the Internet of Things display platform, and optimize the target detection results when the target detection results do not meet the requirements. Furthermore, the specific steps of step S1 are as follows: S11. Obtain the original target detection model based on the YOLOv7 algorithm; S12. Based on the original YOLOv7 algorithm, the multi-branch stacking module is improved, and the feature learning component is introduced to obtain an improved multi-branch stacking module; S13. construct an auxiliary branch for the original target detection model based on the YOLOv7 algorithm, introduce a feature splitting module and a feature fusion module into the auxiliary branch, and use the feature splitting module and the feature fusion module to fuse the improved multi-branch stacking module and the convolution operation, and then superimpose the auxiliary branch with the original target detection model based on the YOLOv7 algorithm to obtain a training network for the target detection model; S14. Use the images collected by the smart camera device and the target detection pictures to build an image dataset, and divide the image dataset into a training set and a test set; S15. Use the training set to train the target detection model of the original YOLOv7 algorithm through the training network of the target detection model, and use the test set for verification after the training is completed.

[0007] Furthermore, the specific steps of step S12 are as follows: S121. By performing different feature extractions in multiple parallel paths; S122. Use feature learning components instead of convolutional learning features; S123. Insert aggregation nodes between layers of different depths to fuse the feature maps of different branches to obtain an improved multi-branch stacking module.

[0008] Furthermore, the specific steps of step S13 are as follows: S131. Obtain the input and output of the original target detection model based on the YOLOv7 algorithm as the input branch and the output branch respectively; S132. Construct an input auxiliary branch using convolution and an improved multi-branch stacking module for the input branch; S133. Construct output auxiliary branches for each output branch using convolution and improved multi-branch stacking modules; S134. Connecting the input of the original target detection model based on the YOLOv7 algorithm to the front end of the input auxiliary branch at the same time; S135. Split each output of the original target detection model based on the YOLOv7 algorithm using a feature splitting module; S136. The splitting result of each feature splitting module is fused with the back end of the input auxiliary branch and the front end of each output auxiliary branch using a feature fusion module to obtain a training network of the target detection model, wherein the input of the original target detection model based on the YOLOv7 algorithm is the input of the training network of the target detection model, and the output of each output auxiliary branch is the output of the training network of the target detection model; The specific steps of step S14 are as follows: S141. Extracting a data source image from an image collected by a smart camera device using a frame extraction method according to a set ratio; S142. Selecting a corresponding number of target images from a public target detection dataset according to the targets and quantities to be detected; S143. Construct an image dataset by using the data source image and the target image, and mark the data source image as the input part, and mark the target image part as the output part; S144. Divide the image dataset into a training set and a test set according to a set ratio.

[0009] Furthermore, the specific steps of step S2 are as follows: S21. The trained object detection model is exported to ONXX format; S22. Remove weights and nodes whose importance is lower than a threshold value from the target detection model in ONXX format by pruning, convert the weights and activation values ​​of the model from floating point numbers to integers with precision lower than the threshold value, and then merge the continuous calculation layers in the target detection model; S23. Generate a model file executable by the smart camera device from the ONXX format file of the optimized target detection model, and send it to the smart camera device.

[0010] Furthermore, the specific steps of step S3 are as follows: S31. Pre-configure the operating environment and self-starting script of the executable model file in the smart camera device, wherein the operating environment includes dependent libraries, drivers and acceleration libraries; S32. The smart camera device uses the OpenCV library in the dependent library to obtain the real-time video stream and converts it into a format that matches the executable model file; S33. After receiving the executable model file, the smart camera device runs the self-starting script and loads the executable model file of the target detection model through the ONNX inference engine; S34. The smart camera device responds to the user's initialization of the reasoning session, receives the user's input and output nodes, calls the reasoning interface to detect and reason about the data source image of the video frame in the real-time video stream, and obtains the reasoning result; S35. The intelligent camera device superimposes the inference result onto the real-time video stream, obtains the video stream detection result and displays it in real time.

[0011] Furthermore, the specific steps of step S4 are as follows: S41. Establish a transmission channel between the smart camera device and the IoT platform through MQTT or HTTP\HTTPS in advance; S42. Configure the network parameters, device ID and authentication key of the smart camera device and register it on the IoT platform; S43. The smart camera device encapsulates the video stream detection result into a message or data packet, and then sends it to the IoT platform through a transmission channel; S44. The IoT platform parses the video stream detection results as messages or data packets and saves them in a database, converts the video stream into image frames, and converts the detection results into JSON format; S45. The IoT platform embeds the video stream and the detection results in the dashboard of the embedded player and provides them to the user for viewing; S46. The IoT platform monitors the detection results to determine whether the fluctuation between the detection results of consecutive frames exceeds a preset continuous threshold; If yes, go to step S47; If not, return to step S3; S47. The IoT platform pushes the video stream to the trained object detection model in the background for object detection, and pushes the detection result to the dashboard of the embedded player for display.

[0012] In a second aspect, the present invention provides a target detection system based on an Internet of Things platform, comprising: A model building unit, used to improve the original target detection model to obtain a target detection model training network, use the target detection model training network, and train the original target detection model in combination with the collected image data set; A model conversion unit, used to export and optimize the trained target detection model, and then convert the optimized target detection model into a model file executable by the smart camera device; The video detection unit is used to load the model file on the smart camera device, use the model file to process the real-time video stream, and obtain the target detection result; The detection display and optimization unit is used to display and monitor the target detection results of the smart camera equipment through the Internet of Things display platform, and optimize the target detection results when the target detection results do not meet the requirements.

[0013] In a third aspect, the present invention provides an electronic device, including a processor and a memory; The memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that the device executes the method described in the first aspect above.

[0014] In a fourth aspect, the present invention provides a storage medium, The storage medium stores instructions, which, when executed on a computer, enable the computer to execute the method described in the first aspect.

[0015] The beneficial effects of the present invention are: The object detection method, system, device and medium based on the Internet of Things platform provided by the present invention enhance the feature expression ability and calculation efficiency of the model by improving the YOLOv7 algorithm and introducing a multi-branch stacking module and a feature learning component, so that the object detection can be efficiently processed in real time on the embedded device and the detection accuracy is improved; the size and calculation amount of the model are effectively reduced by optimization technologies such as model pruning, quantization and layer fusion, the operation efficiency of the model on resource-constrained embedded devices is significantly improved, and the balance between performance and accuracy is ensured; the OpenCV library and ONNX Runtime are used to integrate model reasoning and video stream processing functions, the object detection of the real-time video stream of the camera is realized, and the detection result is superimposed on the video stream for real-time display, providing an intuitive display of the detection result; an efficient data transmission channel between the smart camera and the Internet of Things platform is established by the MQTT or HTTP / HTTPS protocol, and the real-time transmission of the detection result is realized; the Internet of Things platform displays the real-time detection result through the embedded video player and the dashboard, thereby improving the user experience; the quality index of the detection result is set to monitor the received detection result in real time, and when the detection result is not good, the high-precision model on the server side is automatically called for re-detection, the target detection result is optimized, and the stability and reliability of the detection are ensured.

[0016] The present invention improves the real-time performance, accuracy and operating efficiency of the target detection system, improves data transmission and display effects, enhances the stability and reliability of detection results, simplifies system integration and management, and provides strong support for various practical applications.

[0017] It can be seen that compared with the prior art, the present invention has outstanding substantive features and significant progress, and the beneficial effects of its implementation are also obvious. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 It is a flow chart of the target detection method based on the Internet of Things platform of the present invention.

[0020] Figure 2 It is a schematic diagram of a target detection system based on an Internet of Things platform of the present invention.

[0021] Figure 3 It is a structural framework diagram of the multi-branch stacking module I-ELAN of the present invention.

[0022] Figure 4 It is a structural framework diagram of the feature learning component FLA in the multi-branch stacking module I-ELAN of the present invention.

[0023] Figure 5 It is a structural framework diagram of the residual block FLB in the feature learning component FLA of the present invention.

[0024] Figure 6 It is a structural framework diagram of the re-parameterized convolution block FLC in the residual block FLB of the present invention.

[0025] Figure 7 It is a structural framework diagram of the target detection model training network with auxiliary branches of the present invention. DETAILED DESCRIPTION

[0026] The target detection method based on the Internet of Things platform can be applied to one or more devices, which are devices that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions. Its hardware includes but is not limited to microprocessors, application specific integrated circuits (ASICs), programmable gate arrays (FPGAs), digital processors (DSPs), embedded devices, etc.

[0027] A device can be any electronic product that can interact with a user, such as a personal computer, a tablet computer, a smart phone, a personal digital assistant (PDA), an interactive network television (IPTV), etc. The network where the device is located includes but is not limited to the Internet, a wide area network, a metropolitan area network, a local area network, a virtual private network (VPN), etc.

[0028] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0029] See also Figure 1 The figure shows a target detection method based on the Internet of Things platform in a specific implementation, which includes the following steps: S1. Improve the original target detection model to obtain a target detection model training network, use the target detection model training network, and train the original target detection model in combination with the collected image data set; S2. Export and optimize the trained target detection model, and then convert the optimized target detection model into a model file executable by the smart camera device; S3. The smart camera device loads the model file and uses the model file to process the real-time video stream to obtain the target detection result; S4. Display and monitor the target detection results of the smart camera device through the Internet of Things display platform, and optimize the target detection results when the target detection results do not meet the requirements.

[0030] In this embodiment, the specific steps of step S1 are as follows: S11. Obtain the original target detection model based on the YOLOv7 algorithm; S12. Based on the original YOLOv7 algorithm, the multi-branch stacking module is improved, and a feature learning component is introduced to obtain an improved multi-branch stacking module; the specific steps of step S12 are as follows: S121. By performing different feature extractions in multiple parallel paths; S122. Use feature learning components instead of convolutional learning features; S123. Inserting aggregation nodes between layers of different depths to fuse feature maps of different branches to obtain an improved multi-branch stacking module; Improved branch stacking module Figure 3 As shown in the figure, in the improved branch stacking module I-ELAN, the input feature map is subjected to different feature extraction operations through multiple parallel paths, and the feature learning component FLA is used instead of the ordinary convolution learning feature; aggregation nodes are inserted between layers of different depths to fuse the feature maps from different branch paths; Figure 3CBS refers to the convolution operation, c1 to c4 refer to the number of channels of each feature layer, and cat refers to the feature layer concatenation operation. The symbols in the following structure diagram have similar meanings. The improved branch stacking module I-ELAN enhances the feature expression ability of the network through an efficient hierarchical aggregation mechanism while maintaining computational efficiency. The structure of FLA is as follows Figure 4 As shown in the figure, the input feature map is divided into two parts, one part is simply processed by the convolution layer, and the other part is constructed by stacking N residual blocks FLB to build a deeper network. Finally, the outputs of the two paths are spliced ​​and output after convolution processing; stacking N FLBs can improve the network's ability to express features. Each FLB retains a part of the original input features through partial connections across stages, and performs more complex feature extraction in the deep path; The structure of FLB is as follows Figure 5 As shown in the figure, the input feature map is divided into two parts, one part is processed by convolution through the re-parameterized convolution block FLC, and the other part is directly passed to the output end through the jump connection. The two parts of the feature layer are added together through the fusion operation to complete the output; the residual block helps to stabilize the training process, improve the convergence speed, and avoid training difficulties caused by the increase of network depth; The structure of FLC is as follows Figure 6 As shown in the figure, the input feature map is passed to two parallel convolutions at the same time. The kernel size of the first convolution is 3×3, which is used to extract local features and detail information; the kernel size of the second convolution is 1×1, which is used for linear combination and dimensionality reduction between channels; the two feature layers are added together through the fusion operation to complete the output; in the training stage, the FLB structure is a parallel convolution, and in the inference stage, the parallel convolution is re-parameterized into a single convolution to speed up the inference speed and reduce the computational overhead. Without affecting the model performance, the network structure is optimized and the inference efficiency is improved; The improved branch stacking module I-ELAN only uses traditional convolutions, which has a higher parameter utilization rate than advanced deep convolutions and has the great advantages of being lightweight, fast, and accurate. S13. An auxiliary branch is constructed for the original target detection model based on the YOLOv7 algorithm, a feature splitting module and a feature fusion module are introduced into the auxiliary branch, and the feature splitting module and the feature fusion module are used to fuse the improved multi-branch stacking module and the convolution operation, and then the auxiliary branch is superimposed with the original target detection model based on the YOLOv7 algorithm to obtain a training network of the target detection model; the specific steps of step S13 are as follows: S131. Obtain the input and output of the original target detection model based on the YOLOv7 algorithm as the input branch and the output branch respectively; S132. Construct an input auxiliary branch using convolution and an improved multi-branch stacking module for the input branch; S133. Construct output auxiliary branches for each output branch using convolution and improved multi-branch stacking modules; S134. Connecting the input of the original target detection model based on the YOLOv7 algorithm to the front end of the input auxiliary branch at the same time; S135. Split each output of the original target detection model based on the YOLOv7 algorithm using a feature splitting module; S136. The splitting result of each feature splitting module is fused with the back end of the input auxiliary branch and the front end of each output auxiliary branch using a feature fusion module to obtain a training network of the target detection model, wherein the input of the original target detection model based on the YOLOv7 algorithm is the input of the training network of the target detection model, and the output of each output auxiliary branch is the output of the training network of the target detection model; It should be noted that auxiliary branches are added on the basis of the original yolov7 algorithm, such as Figure 7 As shown in the figure; OutputA0, OutputB0, and OutputC0 refer to the three feature layers output by the backbone network of the original yolov7 algorithm; Inputs refers to the input of the backbone network; FLD refers to the feature splitting module, which adjusts the number of channels of its input through convolution and splits it into multiple parts according to the channel dimension; FLE refers to the feature fusion module, which adjusts multiple parts of its input to the same size, adds multiple parts together, and fuses the feature layers; the auxiliary branch processes the feature layers of different scales through feature splitting and feature fusion operations, and outputs three additional feature layers OutputA1, OutputB1, and OutputC1, which can provide reliable gradient information, obtain more effective target task features, and update network parameters; the auxiliary branch is only used in the training stage, which can retain the reasoning ability of the original network and will not increase additional reasoning costs; S14. Use the images collected by the smart camera device and the target detection pictures to build an image data set, and divide the image data set into a training set and a test set; the specific steps of step S14 are as follows: S141. Extract the data source image from the image collected by the smart camera device using the frame extraction method according to the set ratio; for example, take one image every 10 frames to avoid the similarity of adjacent frame images being too high, affecting the quality of the data set, and ensuring the richness of the data set; S142. Selecting a corresponding number of target images from a public target detection dataset according to the targets and quantities to be detected; S143. Construct an image dataset by using the data source image and the target image, and mark the data source image as the input part, and mark the target image part as the output part; S144. Divide the image data set into a training set and a test set according to a set ratio; S15. Train the target detection model of the original YOLOv7 algorithm using the training set through the training network of the target detection model, and verify it using the test set after the training is completed; It should be noted that during the training process, the strategy of training from scratch is used for training, and no pre-training weights are set; The specific steps of step S2 are as follows: S21. The trained object detection model is exported to ONNX format; ONNX supports multiple deep learning frameworks such as TensorFlow, PyTorch, Keras, Caffe, etc. S22. Remove weights and nodes whose importance is lower than a threshold value from the target detection model in ONXX format by pruning, convert the weights and activation values ​​of the model from floating point numbers to integers with precision lower than the threshold value, and then merge the continuous calculation layers in the target detection model; It should be noted that the gradient and activation value indicators are set in advance for the weights or nodes in the target detection model; The importance of each weight or node is evaluated according to the gradient and activation value indicators, and the weights or nodes whose importance is lower than the preset threshold are removed from the target detection model to complete the model pruning; The weights and activation values ​​of the target detection model are mapped from floating point numbers to integer ranges, and during the quantization process, the target detection model is calibrated to determine the final quantization parameters and complete the quantization of the target detection model; Identify the fusible layers in the target detection model, merge the fusible layers into a single layer, and adjust the weights and biases of the target detection model after fusion; The exported ONNX model may contain a lot of redundant information and unoptimized computing nodes. In order to run efficiently on resource-limited embedded devices, the exported model needs to be optimized through model pruning, quantization, and layer fusion. Model pruning can remove unimportant weights and nodes to reduce the size and amount of computation of the model. Model quantization can reduce the use of storage and computing resources. Merging can reduce the storage requirements and computational complexity of intermediate results. S23. Generate an executable model file for the smart camera device from the ONXX format file of the optimized target detection model, and send it to the smart camera device; The specific steps of step S3 are as follows: S31. Pre-configure the operating environment and self-starting script of the executable model file in the smart camera device, wherein the operating environment includes dependent libraries, drivers and acceleration libraries; S32. The smart camera device uses the OpenCV library in the dependent library to obtain the real-time video stream and converts it into a format that matches the executable model file; S33. After receiving the executable model file, the smart camera device runs the self-starting script and loads the executable model file of the target detection model through the ONNX inference engine; S34. The smart camera device responds to the user's initialization of the reasoning session, receives the user's input and output nodes, calls the reasoning interface to detect and reason about the data source image of the video frame in the real-time video stream, and obtains the reasoning result; S35. The smart camera device superimposes the inference result onto the real-time video stream, obtains the video stream detection result and displays it in real time; The specific steps of step S4 are as follows: S41. Establish a transmission channel between the smart camera device and the IoT platform through MQTT or HTTP\HTTPS in advance; S42. Configure the network parameters, device ID and authentication key of the smart camera device and register it on the IoT platform; S43. The smart camera device encapsulates the video stream detection result into a message or data packet, and then sends it to the IoT platform through a transmission channel; S44. The IoT platform parses the video stream detection results as messages or data packets and saves them in a database, converts the video stream into image frames, and converts the detection results into JSON format; S45. The IoT platform embeds the video stream and the detection results in the dashboard of the embedded player and provides them to the user for viewing; S46. The IoT platform monitors the detection results to determine whether the fluctuation between the detection results of consecutive frames exceeds a preset continuous threshold; If yes, go to step S47; If not, return to step S3; It should be noted that the continuous threshold can be set according to the actual detection target and scene; the fluctuation between the detection results of the consecutive frames is set by the target type or the target number; S47. The IoT platform pushes the video stream to the trained object detection model in the background for object detection, and pushes the detection result to the dashboard of the embedded player for display.

[0031] It should be understood that the order of execution of the steps in the above embodiment does not mean that the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0032] The following is an embodiment of a target detection system based on an Internet of Things platform provided in an embodiment of the present disclosure. This system and the target detection system based on an Internet of Things platform in the above-mentioned embodiments belong to the same inventive concept. For details not fully described in the embodiment of the target detection system based on an Internet of Things platform, reference can be made to the embodiment of the target detection method based on the Internet of Things platform mentioned above.

[0033] like Figure 2 As shown, the system includes: A model building unit, used to improve the original target detection model to obtain a target detection model training network, use the target detection model training network, and train the original target detection model in combination with the collected image data set; A model conversion unit, used to export and optimize the trained target detection model, and then convert the optimized target detection model into a model file executable by the smart camera device; The video detection unit is used to load the model file on the smart camera device, use the model file to process the real-time video stream, and obtain the target detection result; The detection display and optimization unit is used to display and monitor the target detection results of the smart camera equipment through the Internet of Things display platform, and optimize the target detection results when the target detection results do not meet the requirements.

[0034] The component comparison method of the electric energy collection device for realizing carbon emission detection provided by the present invention is a combination of the modules and algorithm steps of each example described in the embodiments disclosed in this article, and can be implemented by electronic hardware, computer software or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0035] Those skilled in the art will appreciate that various aspects of the target detection method on the Internet of Things platform provided by the present invention can be implemented as a system, method or program product. Therefore, various aspects of the present disclosure can be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation combining hardware and software, which can be collectively referred to as a "circuit", "module" or "system".

[0036] In the non-transitory computer-readable storage medium of the target detection method stored in the Internet of Things platform, it can be, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0037] Although the present invention has been described in detail with reference to the accompanying drawings and in combination with the preferred embodiments, the present invention is not limited thereto. Without departing from the spirit and essence of the present invention, a person of ordinary skill in the art may make various equivalent modifications or substitutions to the embodiments of the present invention, and these modifications or substitutions shall be within the scope of the present invention. Any person of ordinary skill in the art may easily think of changes or substitutions within the technical scope disclosed by the present invention, and these shall be within the scope of protection of the present invention.

Claims

1. A target detection method based on an Internet of Things platform, characterized in that: The steps include: S1. Improve the original target detection model to obtain a target detection model training network, use the target detection model training network, and train the original target detection model in combination with the collected image data set; S2. Export and optimize the trained target detection model, and then convert the optimized target detection model into a model file executable by the smart camera device; S3. The smart camera device loads the model file and uses the model file to process the real-time video stream to obtain the target detection result; S4. Display and monitor the target detection results of the smart camera device through the Internet of Things display platform, and optimize the target detection results when the target detection results do not meet the requirements.

2. The target detection method based on the Internet of Things platform as claimed in claim 1, characterized in that: The specific steps of step S1 are as follows: S11. Obtain the original target detection model based on the YOLOv7 algorithm; S12. Based on the original YOLOv7 algorithm, the multi-branch stacking module is improved, and the feature learning component is introduced to obtain an improved multi-branch stacking module; S13. construct an auxiliary branch for the original target detection model based on the YOLOv7 algorithm, introduce a feature splitting module and a feature fusion module into the auxiliary branch, and use the feature splitting module and the feature fusion module to fuse the improved multi-branch stacking module and the convolution operation, and then superimpose the auxiliary branch with the original target detection model based on the YOLOv7 algorithm to obtain a training network for the target detection model; S14. Use the images collected by the smart camera device and the target detection pictures to build an image dataset, and divide the image dataset into a training set and a test set; S15. Use the training set to train the target detection model of the original YOLOv7 algorithm through the training network of the target detection model, and use the test set for verification after the training is completed.

3. The target detection method based on the Internet of Things platform as claimed in claim 2, characterized in that: The specific steps of step S12 are as follows: S121. By performing different feature extractions in multiple parallel paths; S122. Use feature learning components instead of convolutional learning features; S123. Insert aggregation nodes between layers of different depths to fuse the feature maps of different branches to obtain an improved multi-branch stacking module.

4. The target detection method based on the Internet of Things platform as claimed in claim 2, characterized in that: The specific steps of step S13 are as follows: S131. Obtain the input and output of the original target detection model based on the YOLOv7 algorithm as the input branch and the output branch respectively; S132. Construct an input auxiliary branch using convolution and an improved multi-branch stacking module for the input branch; S133. Construct output auxiliary branches for each output branch using convolution and improved multi-branch stacking modules; S134. Connecting the input of the original target detection model based on the YOLOv7 algorithm to the front end of the input auxiliary branch at the same time; S135. Split each output of the original target detection model based on the YOLOv7 algorithm using a feature splitting module; S136. The splitting result of each feature splitting module is fused with the back end of the input auxiliary branch and the front end of each output auxiliary branch using a feature fusion module to obtain a training network of the target detection model, wherein the input of the original target detection model based on the YOLOv7 algorithm is the input of the training network of the target detection model, and the output of each output auxiliary branch is the output of the training network of the target detection model; The specific steps of step S14 are as follows: S141. Extracting a data source image from an image collected by a smart camera device using a frame extraction method according to a set ratio; S142. Selecting a corresponding number of target images from a public target detection dataset according to the targets and quantities to be detected; S143. Construct an image dataset by using the data source image and the target image, and mark the data source image as the input part, and mark the target image part as the output part; S144. Divide the image dataset into a training set and a test set according to a set ratio.

5. The target detection method based on the Internet of Things platform as claimed in claim 2, characterized in that: The specific steps of step S2 are as follows: S21. The trained object detection model is exported to ONXX format; S22. Remove weights and nodes whose importance is lower than a threshold value from the target detection model in ONXX format by pruning, convert the weights and activation values ​​of the model from floating point numbers to integers with precision lower than the threshold value, and then merge the continuous calculation layers in the target detection model; S23. Generate a model file executable by the smart camera device from the ONXX format file of the optimized target detection model, and send it to the smart camera device.

6. The target detection method based on the Internet of Things platform as claimed in claim 5, characterized in that: The specific steps of step S3 are as follows: S31. Pre-configure the operating environment and self-starting script of the executable model file in the smart camera device, wherein the operating environment includes dependent libraries, drivers and acceleration libraries; S32. The smart camera device uses the OpenCV library in the dependent library to obtain the real-time video stream and converts it into a format that matches the executable model file; S33. After receiving the executable model file, the smart camera device runs the self-starting script and loads the executable model file of the target detection model through the ONNX inference engine; S34. The smart camera device responds to the user's initialization of the reasoning session, receives the user's input and output nodes, calls the reasoning interface to detect and reason about the data source image of the video frame in the real-time video stream, and obtains the reasoning result; S35. The intelligent camera device superimposes the inference result onto the real-time video stream, obtains the video stream detection result and displays it in real time.

7. The target detection method based on the Internet of Things platform as claimed in claim 6, characterized in that: The specific steps of step S4 are as follows: S41. Establish a transmission channel between the smart camera device and the IoT platform through MQTT or HTTP\HTTPS in advance; S42. Configure the network parameters, device ID and authentication key of the smart camera device and register it on the IoT platform; S43. The smart camera device encapsulates the video stream detection result into a message or data packet, and then sends it to the IoT platform through a transmission channel; S44. The IoT platform parses the video stream detection results as messages or data packets and saves them in a database, converts the video stream into image frames, and converts the detection results into JSON format; S45. The IoT platform embeds the video stream and the detection results in the dashboard of the embedded player and provides them to the user for viewing; S46. The IoT platform monitors the detection results to determine whether the fluctuation between the detection results of consecutive frames exceeds a preset continuous threshold; If yes, go to step S47; If not, return to step S3; S47. The IoT platform pushes the video stream to the trained object detection model in the background for object detection, and pushes the detection result to the dashboard of the embedded player for display.

8. A target detection system based on the Internet of Things platform, characterized in that: include: A model building unit, used to improve the original target detection model to obtain a target detection model training network, use the target detection model training network, and train the original target detection model in combination with the collected image data set; A model conversion unit, used to export and optimize the trained target detection model, and then convert the optimized target detection model into a model file executable by the smart camera device; The video detection unit is used to load the model file on the smart camera device, use the model file to process the real-time video stream, and obtain the target detection result; The detection display and optimization unit is used to display and monitor the target detection results of the smart camera equipment through the Internet of Things display platform, and optimize the target detection results when the target detection results do not meet the requirements.

9. An electronic device, characterized in that: including a processor and a memory; The memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that the device executes the method described in any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium stores instructions, which, when executed on a computer, enable the computer to execute the method described in any one of claims 1 to 7.

Images