Particle motion device based on YOLOv5 and wind power driving, particle tracking method and product

Abstract

The invention provides a particle motion device based on YOLOv5 and wind power driving, a particle tracking method and a product. The device comprises a transparent spherical shell, at least one wind power excitation source and a plurality of spheres arranged in the transparent spherical shell, wherein a plurality of through holes are formed in the surface of the transparent spherical shell. The method is applied to a particle motion device and comprises the following steps: acquiring a real-time video stream containing motion of a sphere in the particle motion device through an image sensor; based on the trained particle tracking detection model, performing feature extraction and recognition on the sphere motion image in the real-time video stream to obtain a particle tracking detection result; wherein the particle tracking detection model is constructed based on a YOLOv5s model. The constructed particle motion device is simple in structure, an expensive high-speed camera and a high-performance processing platform do not need to be adopted, and the complexity and construction cost of a particle motion tracking system can be remarkably reduced.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and particularly to a particle motion device based on YOLOv5 and wind drive, a particle tracking method, and a product. Background Art

[0002] With the rapid development of artificial intelligence technology, object detection and tracking technologies based on deep learning have made remarkable progress and have been widely applied in many fields such as academic analysis, intelligent driving, drones, sports analysis, video surveillance, etc. Deep learning models based on convolutional neural networks (CNNs) and their derivatives have developed rapidly in recent years, and various new models have been continuously proposed, such as: R-CNN, Fast-RCNN, Mask-RCNN, Alex Net, etc. They have high accuracy and real-time performance under ideal conditions. However, their models are too large. In the field of high-speed target tracking, due to the requirements for high-frame-rate video acquisition and powerful computing platforms, these technologies usually require expensive high-speed cameras and high-performance processing platforms, making it impossible to complete too many network deployments in the existing technology and difficult to popularize in many low-cost application scenarios. Summary of the Invention

[0003] The purpose of this application is to provide a particle motion device based on the YOLOv5 model and wind drive, a particle tracking method, and a product, which can at least solve the problems of high complexity and construction cost of the particle motion tracking system in related technologies.

[0004] To solve the above technical problems, in the first aspect of the embodiments of this application, a particle motion device based on the YOLOv5 model and wind drive is provided, including a transparent spherical shell with a plurality of through holes on its surface, at least one wind excitation source, and a plurality of spheres placed inside the transparent spherical shell;

[0005] The transparent spherical shell includes an upper shell, a lower shell spliced to the upper shell, and a support member connected to the lower shell. The upper shell and the lower shell are fixedly connected by screws;

[0006] The surface of the transparent spherical shell is provided with a first through hole, a second through hole, and a third through hole; the first through hole and the second through hole are arranged on the surface of the upper shell, and the normal line of which forms an angle of 45° with the horizontal plane, and the first through hole and the second through hole are symmetric about the axis of the upper shell; the third through hole is arranged on the surface of the lower shell, and its normal line is perpendicular to the horizontal plane; wherein, a metal mesh with a mesh diameter smaller than the diameter of the sphere is covered at the first through hole, the second through hole, and the third through hole; and one of the wind excitation sources is respectively arranged at the first through hole and the second through hole.

[0007] In the second aspect of the embodiments of the present application, a particle tracking method based on the YOLOv5 model and wind power is provided, which is applied to the particle motion device described in the first aspect of the embodiments of the present application, and includes:

[0008] Obtain a real-time video stream containing the motion of the sphere in the particle motion device through an image sensor;

[0009] Extract features and identify the sphere motion images in the real-time video stream based on the trained particle tracking detection model to obtain a particle tracking detection result; wherein, the particle tracking detection model is constructed based on the YOLOv5s model.

[0010] In the third aspect of the embodiments of the present application, a particle tracking system based on the YOLOv5 model and wind power is provided, including:

[0011] An acquisition module for obtaining a real-time video stream containing the motion of the sphere in the particle tracking system through the image sensor;

[0012] A detection module for extracting features and identifying the sphere motion images in the video stream based on the trained particle tracking detection model to obtain a particle tracking detection result; wherein, the particle tracking detection model is constructed based on the YOLOv5s model.

[0013] In the fourth aspect of the embodiments of the present application, an electronic device is provided, including: a memory and a processor, wherein the processor is configured to execute a computer program stored on the memory, and when the processor executes the computer program, each step in the particle tracking method described in the first aspect of the embodiments of the present application is implemented.

[0014] In the fifth aspect of the embodiments of the present application, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, each step in the particle tracking method described in the first aspect of the embodiments of the present application is implemented.

[0015] As can be seen from the above, in the embodiment of the present application, a particle motion device is first constructed. The particle motion device includes a transparent spherical shell with a plurality of openings on its surface, at least one wind force excitation source, and a plurality of spheres placed inside the transparent spherical shell. When the particle motion device operates, the wind force excitation source drives the air flow inside the transparent spherical shell, and the spheres move along with the flowing air. Then, an image sensor is used to obtain a real-time video stream containing the motion of the spheres in the particle motion device. Finally, based on the trained particle tracking and detection model, feature extraction and recognition are performed on the sphere motion images in the real-time video stream to obtain the particle tracking and detection results. Among them, the particle tracking and detection model is constructed based on the YOLOv5s model. The particle motion device constructed by the present invention has a simple structure, does not require expensive high-speed cameras and high-performance processing platforms, and can significantly reduce the complexity and construction cost of the particle motion tracking system.

[0016] It should be understood that the content described in this part is not intended to identify the key or important features of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the related technologies or the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for use in the description of the related technologies or the embodiments of the present application. Obviously, the drawings in the following description are only some embodiments of the present application, rather than all embodiments. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0018] Figure 1 It is a three-view drawing of the spherical shell simulation model of a particle motion device based on the YOLOv5 model and wind power drive provided in the first aspect of the embodiment of the present application;

[0019] Figure 2 It is a schematic diagram of the actual model of a particle motion device based on the YOLOv5 model and wind power drive provided in the first aspect of the embodiment of the present application;

[0020] Figure 3 It is a flowchart of a particle tracking method based on the YOLOv5 model and wind power drive provided in the second aspect of the embodiment of the present application;

[0021] Figure 4 It is a loss curve graph of the final detection model in a particle tracking method based on the YOLOv5 model and wind power drive provided in the second aspect of the embodiment of the present application;

[0022] Figure 5The F1 score image of the final detection model in a particle tracking method based on the YOLOv5 model and wind driving provided in the second aspect of the embodiments of the present application;

[0023] Figure 6 The sample schematic diagram of the dataset in different environments in a particle tracking method based on the YOLOv5 model and wind driving provided in the second aspect of the embodiments of the present application;

[0024] Figure 7 The example diagram of the final recognition effect in a particle tracking method based on the YOLOv5 model and wind driving provided in the second aspect of the embodiments of the present application;

[0025] Figure 8 The refined process schematic diagram of a particle tracking method based on the YOLOv5 model and wind driving provided in the second aspect of the embodiments of the present application;

[0026] Figure 9 The program module schematic diagram of the particle tracking system provided in the third aspect of the embodiments of the present application;

[0027] Figure 10 The module block diagram of the electronic device provided in the fourth aspect of the embodiments of the present application;

[0028] Figure 11 The module block diagram of the computer-readable storage medium provided in the fifth aspect of the embodiments of the present application. Detailed implementation manners

[0029] In order to make the objectives, technical solutions and advantages of the present application more obvious and understandable, the present application will be clearly and completely described below in conjunction with the embodiments of the present application and their accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements with the same or similar functions from beginning to end. It should be understood that the various embodiments of the present application described below are only used to explain the present application and are not used to limit the present application. That is, based on the various embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application. In addition, the technical features involved in the various embodiments of the present application described below can be combined with each other as long as they do not conflict with each other.

[0030] The first aspect of the embodiments of the present application provides a particle motion device based on the YOLOv5 model and wind driving, including a transparent spherical shell with a plurality of through holes on its surface, at least one wind excitation source, and a plurality of spheres placed inside the transparent spherical shell;

[0031] The transparent spherical shell includes an upper shell, a lower shell spliced to the upper shell, and a support member connected to the lower shell. The upper shell and the lower shell are fixedly connected by screws;

[0032] The surface of the transparent spherical shell is provided with a first through hole, a second through hole and a third through hole; the first through hole and the second through hole are arranged on the surface of the upper shell, the included angle between their normal lines and the horizontal plane is 45°, and the first through hole and the second through hole are symmetric about the axis of the upper shell; the third through hole is arranged on the surface of the lower shell, and its normal line is perpendicular to the horizontal plane; wherein, a metal mesh with a mesh diameter smaller than the diameter of the sphere is covered at the first through hole, the second through hole and the third through hole; a wind excitation source is respectively arranged at the first through hole and the second through hole.

[0033] In the process of constructing the particle motion device in this embodiment, software simulation was first carried out using COMSOL. Please refer to Figure 1 , Figure 1 FIG. is the three-view drawing of the spherical shell simulation model of a particle motion device based on the YOLOv5 model and wind drive provided in the first aspect of the embodiment of the present application. The diameter of the spherical shell is 30 cm, and there are three through holes on the surface: the first through hole K1, the second through hole K2 and the third through hole K3. The diameters of the three through holes are all 12 cm. The whole spherical shell is regarded as two upper and lower hemispheres. The first through hole K1 and the second through hole K2 are arranged on the surface of the upper shell, the included angle between their normal lines and the horizontal plane is 45°, and the first through hole K1 and the second through hole K2 are symmetric about the axis of the upper shell. The third through hole K3 is arranged on the surface of the lower shell, and its normal line is perpendicular to the horizontal plane. In the simulation, the boundary conditions of K1 and K2 are fans, and the air flow direction is: from the inside to the outside; the boundary condition of K3 is the inlet, and the air flow direction is: from the outside to the inside. After the fan is turned on, a flow field is formed inside the spherical shell, driving the particles therein to move. The properties of the particles in the simulation are: diameter 1 cm, density 0.04 g / cm3, and the material is foam.

[0034] After the simulation is completed, the embodiment of the present application customized the actual model according to the simulation results. Please refer to Figure 2 , the same as the above simulation model are the size of the spherical shell, the through hole position, the boundary conditions, and the properties of the sphere. Specifically, the diameter of the actual transparent spherical shell model is 30 cm, the material is transparent acrylic, and there are two fans of the same model outside the two openings at K1 and K2. The wind direction is from the inside to the outside, and the rotation speed is adjustable; the average diameter of the sphere is about 1 cm, the material is lightweight foam plastic, with three colors of red, yellow and blue, and the average density is about 0.04 g / cm 3. The actual model is different from the simulation model in the following ways: a piece of metal mesh is pasted at K1, K2 and K3 to prevent the ball from being drawn into the fan blades and falling out of the spherical shell; the entire spherical shell is divided into two upper and lower hemispheres, the contact surfaces of the two hemispheres extend outward from the center of the sphere, and the extended area is fixed with screws to make the two hemispheres fit together. At the same time, threads are provided on the extended area for installing acrylic rods to support the entire sphere. It can be understood that the shape of the spherical shell of the particle tracking device in the embodiment of the present application, the number of wind excitation sources, the number and position of through holes can be adaptively modified according to actual needs, so as to establish other airflow motion models. Furthermore, the excitation source that drives the motion of the sphere can also be changed so that it is no longer driven by airflow. The particle tracking decorative structure established in the embodiment of the present application is simple and low-cost, and does not require the use of expensive high-speed cameras and high-performance processing platforms, which can significantly reduce the complexity and construction cost of the particle motion tracking system.

[0035] See also Figure 3 , Figure 3 A flow chart of a particle tracking method based on a YOLOv5 model and wind drive provided in the second aspect of an embodiment of the present application, wherein the particle tracking method is applied to the particle motion device provided in the first aspect of the embodiment of the present application, and comprises the following steps.

[0036] Step 301: Acquire a real-time video stream including the motion of a sphere in a particle motion device through an image sensor.

[0037] Step 302: Based on the trained particle tracking detection model, feature extraction and recognition are performed on the spherical motion image in the real-time video stream to obtain a particle tracking detection result; wherein the particle tracking detection model is constructed based on the YOLOv5s model.

[0038] Specifically, the YOLOv5 model is deployed on NVIDIA Jetson nano, and the sphere moving in the spherical shell is tracked and tested through the IMX219 image sensor on Jetson nano. Preferably, as an embedded platform with low computing power, jetson nano needs to use TensorRT to accelerate forward reasoning optimization of the improved YOLOv5s model deployed on Jetson Nano in order to achieve real-time detection. Among them, the IMX219 image sensor is connected to Jetson Nano through the CSI interface; the image sensor reads the input video stream of the ball movement in the spherical shell in real time through OpenCV, and sends it to the particle tracking detection model built based on YOLOv5s for reasoning, and finally obtains the particle tracking detection result.

[0039] In the embodiment of the present application, a particle motion device is first constructed. The particle motion device includes a transparent spherical shell with multiple openings on its surface, at least one wind force excitation source, and multiple spheres placed inside the transparent spherical shell. After the particle motion device operates, the wind force excitation source drives the air flow inside the transparent spherical shell, and the spheres move along with the flowing air. Then, an image sensor is used to obtain a real-time video stream containing the motion of the spheres in the particle motion device. Finally, based on the trained particle tracking and detection model, feature extraction and recognition are performed on the sphere motion images in the real-time video stream to obtain the particle tracking and detection results. Among them, the particle tracking and detection model is constructed based on the YOLOv5s model. The particle motion device constructed by the present invention has a simple structure and does not require expensive high-speed cameras and high-performance processing platforms, which can significantly reduce the complexity and construction cost of the particle motion tracking system.

[0040] In an alternative embodiment of this embodiment, before the step of performing feature extraction and recognition on the images in the real-time video stream based on the trained particle tracking and detection model, the following steps are further included:

[0041] Construct a data set containing multiple sphere motion image samples, and divide the data set into a training sample set and a validation sample set according to a preset ratio;

[0042] Input the sphere motion image samples in the training sample set into the original particle tracking and detection model for object detection to obtain training results;

[0043] Adjust the network parameters in the original particle tracking and detection model based on the training results and a preset loss function to obtain an adjusted particle tracking and detection model; wherein, the preset loss function includes a classification loss function, a localization loss function, and a confidence loss function;

[0044] Input the sphere motion image samples in the validation sample set into the adjusted particle tracking and detection model for object recognition to obtain validation results;

[0045] If the validation is passed, the adjusted particle tracking and detection model is used as the trained particle tracking and detection model; if the validation fails, the adjusted particle tracking and detection model is continuously trained.

[0046] In the embodiment of the present application, the constructed data set contains spheres of three colors, and all the pictures in the data set are in jpg format. The data set is divided into a training sample set and a validation sample set according to a ratio of 9:1. The model is trained with the training sample set, the network parameters in the model are adjusted according to the training results and the loss function, and finally the adjusted model is verified with the validation sample set. The model that passes the verification can be used as the trained model for subsequent reference.

[0047] As a preferred embodiment, the weight file obtained by training the YOLOv5s model on the COCO dataset first is used as the initial training weight. Through transfer learning, the training efficiency is improved, the model performance is enhanced, the model convergence is accelerated, and the generalization ability and robustness of the model are strengthened. Then, further training is carried out on the self-built dataset. Specifically, the training device in the embodiment of the present application is GeForce RTX2060.

[0048] In an alternative embodiment of the present application, the step of inputting the sphere motion image samples in the training sample set into the original particle tracking detection model for target detection to obtain the training result includes:

[0049] Input the sphere motion image samples in the training sample set into the original particle tracking detection model and perform target detection with different numbers of rounds, and obtain the training results with 100 rounds, 150 rounds, 200 rounds, and 250 rounds of training respectively;

[0050] If the verification is passed, the step of using the adjusted particle tracking detection model as the trained particle tracking detection model includes:

[0051] If there are multiple verification results with different numbers of training rounds that are all verified to be passed at the same time, use the adjusted particle tracking detection model with the highest verification accuracy as the trained particle tracking detection model.

[0052] As a preferred embodiment, aiming at the problems of small model depth and limited data volume of the dataset, in order to obtain the model with the optimal detection performance and avoid misdetection caused by model overfitting, the YOLOv5s model is trained using different training strategies and setting different numbers of training rounds. The specific training steps are as follows: Select 100 rounds, 150 rounds, 200 rounds, and 250 rounds of training rounds respectively, and select the model with the highest accuracy verified by the validation set among all the experimental results as the final detection model. Specifically, Figure 4 is the loss curve graph of the final detection model, Figure 5 is the F1 score image.

[0053] In an alternative embodiment of this example, the step of constructing a dataset containing multiple sphere motion image samples includes:

[0054] Run the particle motion device;

[0055] When the sphere motion in the particle motion device is stable, shoot the video containing the sphere motion in the particle motion device and obtain the single-frame images in the video;

[0056] Construct a dataset containing multiple sphere motion image samples based on the single-frame images in the video.

[0057] Specifically, there are three spheres with different colors in the particle motion device. The average diameter of the spheres is about 1 cm, and the material is lightweight foam plastic, with three colors: red, yellow, and blue. The average density is about 0.04 g / cm3. After turning on the wind excitation source (fan), it will drive the air flow inside the spherical shell. Due to the physical properties of the spheres, they will move in the spherical shell along with the air flow. When the movement of the spheres stabilizes, a camera or image sensor is used to shoot a video at 1080P 60 frames, import it into the computer, obtain each frame of the image, and manually annotate it through annotation software to form a data set. And the data set is divided into a training set and a validation set. The training set is used to train the model, and the validation set is used to verify the trained model. This data set captures the real dynamic movement of the small balls under different wind speeds and lighting conditions through shooting, which can reflect the complexity of the environment in the real world and improve the generalization ability of the data set. By establishing such a tracking system and data set, it provides a new experimental platform for turbulence analysis research, and also provides new ideas for object motion detection in the fields of fluid dynamics, aerodynamics, etc., with potential application value.

[0058] In an alternative embodiment of this embodiment, the step of shooting a video containing the movement of the spheres in the particle motion device includes:

[0059] Adjust the shooting background, lighting conditions, and shooting angle of the particle motion device multiple times, and shoot videos containing the movement of the spheres in the particle motion device under different environments.

[0060] To enhance the generalization of the trained model, photos of the small ball movement taken from different environments, different lighting conditions, and different angles are used as data set samples. Refer to Figure 6 , which are respectively natural light in the natural environment, double-stage lamp lighting in the black background environment, single-stage lamp lighting in the black background environment; photos of the small ball movement with natural light in the natural environment, double-stage lamp lighting in the black background environment, and single-stage lamp lighting in the black background environment at another angle. Manually screen and delete the pictures where the small balls are all invisible. Finally, a total of 1024 pictures are obtained. The data set established through shooting can reflect the complex dynamics of fast-moving objects in actual applications, provides new ideas for object motion monitoring in the fields of fluid dynamics, aerodynamics, etc., provides solid data support for object tracking algorithms, and promotes the application progress of object tracking technology in the actual environment. It is a representative and challenging data set.

[0061] In an alternative embodiment of this embodiment, the sphere motion image samples are pre-processed and annotated through LableImg annotation software; among them, the annotation contains visibility information, and the visibility information is used to represent the recognition difficulty of the spheres in the sphere motion image samples;

[0062] The preset loss function also includes a dynamic weight allocation function, which is used to return different weights according to the visibility information and adjust the classification loss function, the localization loss function, and the confidence loss function simultaneously according to the returned weights.

[0063] In addition to performing common preprocessing operations, the embodiments of the present application introduce a new "visibility" label. By modifying the YOLOv5 source code, the annotation file format that YOLO can recognize is changed, and the "visibility" label is added to dynamically adjust the loss weights of difficult-to-recognize targets, helping to improve the generalization of model training.

[0064] Specifically, the visibility of the small ball has the following situations:

[0065] (0) Clearly visible;

[0066] (1) The deformation of the small ball is observed due to the low camera shooting frame rate;

[0067] (2) The semi-transparent area of the spherical shell, the blurring of the edge of the small ball caused by light, and the reduction of visibility;

[0068] (3) The small ball is partially blocked by the opaque area of the spherical shell or other small balls;

[0069] And combinations of the above situations. When a difficult-to-recognize small ball appears, check "difficult" in the LabelImg software, and add suffixes according to the three categories of reasons for difficulty in recognition, which are: 0, 1, 2, 3, 1_2, 1_3, 2_3, 1_2_3, and their numbers are: 774, 73, 90, 274, 272, 172, 353. After the annotation is completed, the LabelImg software will generate an annotation file in xml format, and then convert it into a txt file that YOLO can recognize through a python script.

[0070] The txt file that the original YOLO can recognize contains information in the following format: <class name><x coordinate><y coordinate><width><height>. When converting the xml annotation file into a txt file, extract the label suffix as the visibility information and add it to the txt file. Therefore, the new txt file format is: <class name><x coordinate><y coordinate><width><height><visibility>. The <visibility> information will adjust the training weights according to the number of times the suffix appears and the difficulty of target recognition.

[0071] Finally, in the embodiments of this application, the LabelImg annotation software is used to annotate three types of small balls in the dataset, and the label names are blue ball, red ball, and yellow ball respectively. A total of 2660 targets are found in all the pictures, and 2660 labels are annotated.

[0072] Furthermore, by modifying loss.py in the yolov5 source code to dynamically allocate weights, compared with the original code for loss calculation, a get_difficult_weight function is additionally defined. Its function is to return different weights according to different visibility information to simultaneously adjust the three loss functions (classification loss, localization loss, confidence loss), and finally achieve the effect of making the model pay more attention to difficult-to-identify targets. The specific allocation method is as follows:

[0073]

[0074]

[0075] Finally, three spheres of different colors are detected by the particle tracking model in the embodiments of this application. An example diagram of the recognition effect is as Figure 7 shown.

[0076] In summary, on the one hand, in the embodiments of this application, a particle motion device is first constructed. The particle motion device includes a transparent spherical shell with multiple openings on its surface, at least one wind excitation source, and multiple spheres placed inside the transparent spherical shell. When the particle motion device operates, the wind excitation source drives the air flow inside the transparent spherical shell, and the spheres will move along with the flowing air. Then, a real-time video stream containing the sphere motion in the particle motion device is obtained through an image sensor. Finally, based on the trained particle tracking detection model, feature extraction and recognition are performed on the sphere motion images in the real-time video stream to obtain the particle tracking detection result. Among them, the particle tracking detection model is constructed based on the YOLOv5s model. The particle motion device constructed by the present invention has a simple structure and does not require expensive high-speed cameras and high-performance processing platforms, which can significantly reduce the complexity and construction cost of the particle motion tracking system. On the other hand, the embodiments of this application also establish a dataset through shooting that can reflect the complex dynamics of fast-moving objects in actual applications, providing new ideas for object motion monitoring in fields such as fluid dynamics and aerodynamics, providing solid data support for object tracking algorithms, and promoting the application progress of object tracking technology in actual environments. It is a representative and challenging dataset. In addition, by modifying the YOLOv5 source code, the embodiments of this application change the annotation file format that YOLO can recognize, add a "visibility" label, and dynamically adjust the loss weights for difficult-to-identify targets with the "visibility" label to help improve the generalization of model training.

[0077] It should be understood that the magnitudes of the sequence numbers of the steps in this embodiment do not indicate the sequence of step execution. The execution sequence of each step should be determined by its function and internal logic, and should not uniquely limit the implementation process of the embodiments of this application.

[0078] In summary, Figure 8 The figure is a schematic diagram of the refined process of a particle tracking method based on the YOLOv5 model and wind power drive provided by the embodiments of this application, specifically as follows:

[0079] Step 801: Establish and run a particle motion device;

[0080] Step 802: After the sphere in the particle motion device moves stably, adjust the shooting background, lighting conditions, and shooting angle of the image sensor of the particle motion device multiple times, and shoot a video containing the movement of the sphere in the particle motion device under different environments;

[0081] Step 803: Construct a data set containing multiple sphere motion image samples based on the single-frame images in the video, and add visibility information to the annotations of the sphere motion image samples through the LableImg annotation software;

[0082] Step 804: Divide the data set into a training sample set and a validation sample set according to a preset ratio;

[0083] Step 805: Train the original particle tracking detection model based on the training sample set, and adjust the network parameters in the original particle tracking detection model according to the training results and a preset loss function to obtain an adjusted particle tracking detection model;

[0084] Step 806: Input the sphere motion image samples in the validation sample set into the adjusted particle tracking detection model for target recognition to obtain a validation result;

[0085] Step 807: If the validation passes, use the adjusted particle tracking detection model as the trained particle tracking detection model; if the validation fails, continue to train the adjusted particle tracking detection model;

[0086] Step 808: Obtain a real-time video stream containing the movement of the sphere in the particle motion device through the image sensor;

[0087] Step 809: Extract features and recognize the sphere motion images in the real-time video stream based on the trained particle tracking detection model to obtain a particle tracking detection result.

[0088] For the more detailed processes of each step in Steps 801 to 809, refer to the descriptions of the relevant parts shown above. The embodiments of this application will not elaborate further here.

[0089] Please refer to Figure 9 , Figure 9 which is a schematic diagram of the program modules of the particle tracking system provided in the third aspect of the embodiments of the present application. This system can be used to implement the particle tracking method involved in the embodiments of the present application. The particle tracking system mainly includes:

[0090] An acquisition module 901, configured to acquire a real-time video stream containing the movement of the sphere in the particle tracking system through an image sensor;

[0091] A detection module 902, configured to perform feature extraction and recognition on the sphere movement images in the video stream based on a trained particle tracking detection model to obtain a particle tracking detection result; wherein, the particle tracking detection model is constructed based on the YOLOv5s model.

[0092] In some embodiments of this embodiment, before the detection module 902 executes the step of performing feature extraction and recognition on the images in the real-time video stream based on the trained particle tracking detection model, it further includes a training module, which is configured to: construct a data set containing multiple sphere movement image samples, and divide the data set into a training sample set and a validation sample set according to a preset ratio; input the sphere movement image samples in the training sample set into the original particle tracking detection model for object detection to obtain a training result; adjust the network parameters in the original particle tracking detection model based on the training result and a preset loss function to obtain an adjusted particle tracking detection model; wherein, the preset loss function includes a classification loss function, a localization loss function, and a confidence loss function; input the sphere movement image samples in the validation sample set into the adjusted particle tracking detection model for object recognition to obtain a validation result; if the validation passes, then use the adjusted particle tracking detection model as the trained particle tracking detection model; if the validation fails, then continue to train the adjusted particle tracking detection model.

[0093] Furthermore, in some embodiments of this embodiment, when the training module executes the step of inputting the sphere movement image samples in the training sample set into the original particle tracking detection model for object detection to obtain a training result, it is specifically configured to: input the sphere movement image samples in the training sample set into the original particle tracking detection model and perform object detection with different training rounds to obtain training results with training rounds of 100 rounds, 150 rounds, 200 rounds, and 250 rounds respectively; when the training module executes the step of if the validation passes, then use the adjusted particle tracking detection model as the trained particle tracking detection model, it is specifically configured to: if there are multiple validation results with different training rounds that are all validation passed at the same time, then use the adjusted particle tracking detection model with the highest validation accuracy as the trained particle tracking detection model.

[0094] In some embodiments of this embodiment, when the training module executes the step of constructing a data set containing multiple sphere motion image samples, it is specifically used to: run the particle motion device; when the sphere motion in the particle motion device is stable, capture a video containing the sphere motion in the particle motion device, and obtain a single-frame image from the video; construct a data set containing multiple sphere motion image samples based on the single-frame image in the video.

[0095] Further, in some embodiments of this embodiment, when the training module executes the step of capturing a video containing the sphere motion in the particle motion device, it is specifically used to: adjust the shooting background, lighting conditions, and shooting angle of the particle motion device multiple times, and capture a video containing the sphere motion in the particle motion device under different environments.

[0096] In some embodiments of this embodiment, the sphere motion image samples are pre-processed and labeled by the LableImg labeling software; wherein, the labeling contains visibility information, and the visibility information is used to represent the recognition difficulty of the spheres in the sphere motion image samples; the preset loss function further includes a dynamic weight assignment function, and the dynamic weight assignment function is used to return different weights according to the visibility information, and adjust the classification loss function, localization loss function, and confidence loss function simultaneously according to the returned weights.

[0097] The particle tracking system provided in this embodiment is applied to the particle motion device provided in the first aspect of the embodiments of the present application. The particle motion device includes a transparent spherical shell with multiple openings on its surface, at least one wind excitation source, and multiple spheres placed inside the transparent spherical shell. When the particle motion device runs, the wind excitation source drives the air flow inside the transparent spherical shell, and the spheres move along with the flowing air. Then, a real-time video stream containing the sphere motion in the particle motion device is obtained through an image sensor. Finally, feature extraction and recognition are performed on the sphere motion images in the real-time video stream based on the trained particle tracking detection model to obtain the particle tracking detection result. Among them, the particle tracking detection model is constructed based on the YOLOv5s model. The particle motion device constructed by the present invention has a simple structure, does not require expensive high-speed cameras and high-performance processing platforms, and can significantly reduce the complexity and construction cost of the particle motion tracking system.

[0098] Please refer to Figure 10 , Figure 10 which is a block diagram of the modules of the electronic device provided in the embodiments of the present application.

[0099] As Figure 10As shown in the figure, an embodiment of the present application further provides an electronic device, which can be used to implement the particle tracking method in the foregoing embodiments. The electronic device includes a memory 1001 and at least one processor 1002. Among them, the memory 1001 is used to store at least one program, and when the at least one program is executed by the at least one processor 1002, the at least one processor 1002 executes the particle tracking method provided by the embodiment of the present application.

[0100] Please refer to Figure 11 , Figure 11 which is a block diagram of a module of a computer-readable storage medium provided by an embodiment of the present application.

[0101] As Figure 11 shown in the figure, an embodiment of the present application further provides a computer-readable storage medium 1100. An executable instruction 1110 is stored on the computer-readable storage medium 1100. When the executable instruction 1110 is executed, the particle tracking method provided by the embodiment of the present application is executed.

[0102] The steps of the method or algorithm described in combination with the embodiments disclosed in this document can be directly implemented by hardware, a software module executed by a processor, or a combination of the two. The software module can be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, register, hard disk, removable disk, CD-ROM, or any other form of storage medium well known in the technical field.

[0103] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in accordance with the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center in a wired (such as coaxial cable, optical fiber, digital subscriber line) or wireless (such as infrared, wireless, microwave, etc.) manner. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more integrated available media. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk).

[0104] It should be noted that the various embodiments in the content of this application are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same or similar parts among the embodiments, reference can be made to each other. For product embodiments, since they are similar to method embodiments, the description is relatively simple, and reference can be made to the corresponding parts of the method embodiments for relevant content.

[0105] It should also be noted that in the content of this application, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising one..." does not exclude the existence of additional identical elements in the process, method, article or device comprising the said element.

[0106] The above description of the disclosed embodiments enables those skilled in the art to implement or use the content of this application. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined in the content of this application can be implemented in other embodiments without departing from the spirit or scope of the content of this application. Therefore, the content of this application will not be limited to these embodiments shown in the content of this application, but rather to the broadest scope consistent with the principles and novel features disclosed in the content of this application.

Claims

1. A particle motion device based on the YOLOv5 model and wind-driven, characterized in that It includes a transparent spherical shell with multiple through holes on its surface, at least one wind excitation source, and multiple spheres placed inside the transparent spherical shell; The transparent spherical shell includes an upper shell, a lower shell spliced to the upper shell, and a support member connected to the lower shell. The upper shell and the lower shell are fixedly connected by screws; The surface of the transparent spherical shell is provided with a first through hole, a second through hole, and a third through hole; the first through hole and the second through hole are arranged on the surface of the upper shell, and the included angle between their normal lines and the horizontal plane is 45°, and the first through hole and the second through hole are symmetric about the axis of the upper shell; the third through hole is arranged on the surface of the lower shell, and its normal line is perpendicular to the horizontal plane; wherein, a metal mesh with a mesh diameter smaller than the diameter of the sphere is covered at the first through hole, the second through hole, and the third through hole; one of the wind excitation sources is respectively arranged at the first through hole and the second through hole.

2. A particle tracking method based on the YOLOv5 model and wind-driven, characterized in that, Applied to the particle motion device described in claim 1, it includes: Obtain a real-time video stream containing the motion of the spheres in the particle motion device through an image sensor; Extract and identify the features of the sphere motion images in the real-time video stream based on a trained particle tracking and detection model to obtain a particle tracking and detection result; wherein, the particle tracking and detection model is constructed based on the YOLOv5s model.

3. The particle tracking method according to claim 2, wherein Before the step of extracting and identifying the features of the images in the real-time video stream based on the trained particle tracking and detection model, it further includes: Construct a data set containing multiple sphere motion image samples, and divide the data set into a training sample set and a validation sample set according to a preset ratio; Input the sphere motion image samples in the training sample set into the original particle tracking and detection model for object detection to obtain a training result; Adjust the network parameters in the original particle tracking and detection model based on the training result and a preset loss function to obtain an adjusted particle tracking and detection model; wherein, the preset loss function includes a classification loss function, a localization loss function, and a confidence loss function; Input the sphere motion image samples in the validation sample set into the adjusted particle tracking and detection model for object recognition to obtain a validation result; If the validation passes, then use the adjusted particle tracking and detection model as the trained particle tracking and detection model; if the validation fails, then continue to train the adjusted particle tracking and detection model.

4. The particle tracking method according to claim 3, wherein The step of inputting the sphere motion image samples in the training sample set into the original particle tracking and detection model for object detection to obtain a training result includes: Input the sphere motion image samples in the training sample set into the original particle tracking and detection model and perform object detection with different training epochs, respectively obtaining training results with training epochs of 100, 150, 200, and 250; The step of if the validation passes, then use the adjusted particle tracking and detection model as the trained particle tracking and detection model includes: If there are multiple verification results with different numbers of training rounds that are simultaneously verified as passed, the adjusted particle tracking detection model with the highest verification accuracy is used as the particle tracking detection model that has completed training.

5. The particle tracking method according to claim 3, characterized in that, The step of constructing a data set containing multiple sphere motion image samples includes: Running the particle motion device; When the sphere motion in the particle motion device is stable, taking a video containing the sphere motion in the particle motion device and obtaining single-frame images in the video; Constructing a data set containing multiple sphere motion image samples based on the single-frame images in the video.

6. The particle tracking method according to claim 5, characterized in that The step of taking a video containing the sphere motion in the particle motion device includes: Adjusting the shooting background, lighting conditions, and shooting angle of the particle motion device multiple times and taking videos containing the sphere motion in the particle motion device under different environments.

7. The particle tracking method according to claim 5, characterized in that The sphere motion image samples are pre-processed and labeled by the LableImg labeling software; wherein, the labeling contains visibility information, and the visibility information is used to represent the recognition difficulty of the spheres in the sphere motion image samples; The preset loss function further includes a dynamic weight allocation function, and the dynamic weight allocation function is used to return different weights according to the visibility information and simultaneously adjust the classification loss function, the localization loss function, and the confidence loss function according to the returned weights.

8. A particle tracking system based on the YOLOv5 model and wind-driven, characterized in that, Including: An acquisition module, configured to acquire a real-time video stream containing the sphere motion in the particle tracking system through an image sensor; A detection module, configured to perform feature extraction and recognition on the sphere motion images in the video stream based on the trained particle tracking detection model to obtain a particle tracking detection result; wherein, the particle tracking detection model is constructed based on the YOLOv5s model.

9. An electronic device, characterized in that, Including a memory and a processor, wherein: The processor is configured to execute a computer program stored on the memory; When the processor executes the computer program, the steps in the particle tracking method according to any one of claims 2 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the steps in the particle tracking method according to any one of claims 2 to 7 are implemented.

Images