Underwater dynamic target detection and identification method based on C # upper computer and ESP32CAM

Abstract

The invention discloses an underwater dynamic target detection and recognition method based on a C # upper computer and an ESP32CAM, and particularly relates to the technical field of underwater dynamic target detection and recognition, which comprises the following steps: step 1, firstly, carrying out initialization operation on the ESP32CAM, setting key parameters, defining the packet segmentation size of a sent picture, enabling each image packet to have 1024 bytes at most, and carrying out data processing on the ESP32CAM; the size and the resolution of a sampled image are set to be 800 * 600, the quality of the image is set, the smaller the 10-63 is, the best the quality of the image is, if the color of the image is strange or the image is split into several parts, the numerical value needs to be increased, and one image is collected each time so as to ensure that the quality and the fluency of the collected image meet the requirements. According to the invention, the captured underwater image signal can be transmitted to the upper computer system written based on C # in real time through the camera WIFI communication of the ESP32CAM, the real-time performance and accuracy of data transmission are ensured, the underwater dynamic target can be detected and identified in real time, and the actual use effect is good.

Description

Technical Field

[0001] The present invention relates to the technical field of underwater dynamic target detection and recognition, and more specifically, to an underwater dynamic target detection and recognition method based on a C# host computer and ESP32CAM. Background Art

[0002] With the rapid advancement of science and technology and the continuous deepening of human exploration of nature, humans have gradually relied on the use of automated unmanned underwater equipment to assist in a series of underwater operations in the ocean. For example, unmanned underwater fishing devices are used to automatically catch fish and shellfish, avoiding the dangers and physical harm caused by manual diving fishing, while improving fishing efficiency. Underwater cleaning robots can be used to salvage marine garbage and purify the marine environment. The use of underwater equipment is inseparable from the underwater target recognition system. When performing underwater tasks, the robot needs to collect a variety of information including underwater images, water quality data, and acoustic signals in order to conduct comprehensive and accurate analysis and recognition of the underwater environment. The existing underwater target recognition detection method cannot achieve real-time detection and recognition of underwater dynamic targets, so the actual use effect is poor.

[0003] Therefore, there is an urgent need for an underwater dynamic target detection and recognition method based on C# host computer and ESP32CAM to solve the above problems. Summary of the invention

[0004] In order to overcome the above-mentioned defects of the prior art, the present invention provides an underwater dynamic target detection and recognition method based on a C# host computer and ESP32CAM. The present invention can transmit the captured underwater image signal in real time to a host computer system written in C# through the camera WIFI communication of ESP32CAM, thereby ensuring the real-time and accuracy of data transmission, and can detect and identify underwater dynamic targets in real time. The actual use effect is good, and the problems raised in the above-mentioned background technology are solved.

[0005] To achieve the above object, the present invention provides the following technical solution: a method for underwater dynamic target detection and recognition based on a C# host computer and ESP32CAM, comprising the following steps: Step 1: First, initialize ESP32-CAM, set key parameters, define the packet size for sending pictures, each image packet is up to 1024 bytes, set the sampled image size and resolution to 800*600, set the quality of the picture, the smaller the value of 10-63, the better the quality of the picture. If the picture color is strange or split into several parts, you need to increase the value. Collect one picture at a time to ensure that the quality and fluency of the collected image meet the requirements; Step 2: ESP32-CAM connects to the specified WIFI network. If the connection is successful, it will obtain its own IP address in the network and prepare for subsequent data transmission; Step 3: ESP32-CAM will establish a UDP connection with the host computer. The UDP protocol is very suitable for the transmission of real-time image data due to its high efficiency and lightweight. Through UDP transmission, ESP32-CAM can send the collected image data to the host computer in real time and quickly; Step 4: ESP32-CAM enters the loop acquisition mode, which continuously captures new image data from the camera and transmits this data to the host computer in real time through the established UDP communication. This data arrives in the form of a byte stream and needs to be properly decoded and format converted for subsequent processing. Step 5: The server uses C# to write a host computer program, which is responsible for receiving the image data sent by the client and performing subsequent processing and recognition work. C# has good readability, strong ease of use, and is cross-platform. The host computer program loads the pre-trained YOLOv5n model and uses the loaded YOLOv5n model to detect and recognize underwater dynamic objects in the received images. This process includes image preprocessing, model reasoning and post-processing, and finally outputs the recognition results; Step 6: Display the recognition results to the user in a visual form by displaying the annotated recognition image through the PictureBox control.

[0006] In a preferred embodiment, the host computer software C# is mainly responsible for the following: (a) Image data reception: responsible for receiving real-time image data stream from the ESP32-CAM device; (b) Image decoding and display: Decode the image data received in the previous step and display the decoded image in real time on the software interface; (c) Object detection processing: The decoded image data is passed to the background YOLOv5n object detection model for processing. This process involves format conversion and preprocessing of the image data to meet the input requirements of the YOLOv5n model. (d) Display and save detection results: Receive the target detection results returned by the YOLOv5n model, mark the detected targets on the original image, and save these results to a local file for subsequent analysis and use.

[0007] In a preferred embodiment, the UDP is used to complete the reception and analysis of the wireless communication module on the data packet of the wireless network, and the code to implement this function includes: server_client = newSocket(AddressFamily.InterNetwork,SocketType.Dgram,ProtocolType.Udp), create a UDP protocol Socket, use the IPv4 protocol family, datagram type Socket type; server_client.Bind(newIPEndPoint(IPAddress.Parse(this.textBox1.Text),int.Parse(this.textBox2.Text))), binds the Socket to the local IP address and port, and uses the two text boxes (textBox1 and textBox2) in the form to obtain the IP address and port number; (c) RThread = new Thread(ReceiveMsg), creates a thread (RThread) to execute the logic of receiving UDP packets; (d) EndPoint point = new IPEndPoint(IPAddress.Any, 0), used to receive data from the sender; (e) int length = server_client.ReceiveFrom(bufferCatch, ref point), get socket data; (f) if ((bufferCatch[0] =='o') && (bufferCatch[1] =='k')), when the ok message is received, it means that the next packet will be received immediately, and the last transmitted image information has been received.

[0008] In a preferred embodiment, the WiFi module configuration includes: (a) WiFi.mode(WIFI_STA), connect to WiFi, the mode is STA mode; (b) WiFi.begin(ssid,password), connect to the router, ssid is the name of the connected WIFI, password is the password of the connected WIFI; (c) If the WIFI connection is successful, the local IP address will be printed out according to the instructions. If the connection is unsuccessful, a failed connection will appear and a new connection will be established after 1000 milliseconds.

[0009] In a preferred embodiment, the UDP configuration includes: (a) if (udp.connect(IPAddress(192,168,43,14),8080)), use the WIFI module to realize the UDP communication connection with the host computer, where 192.168.43.14 is the IP address of the host computer and 8080 is the port number of the host computer; (b) Serial.println("UDP connected"), if UDP is successfully connected, the characters "UDP connected" will be printed out.

[0010] Technical effects and advantages of the present invention: The present invention first initializes ESP32-CAM, sets key parameters, defines the packet segmentation size of the sent picture, each image packet is 1024 bytes at most, sets the sampled picture size and resolution to 800*600, sets the quality of the picture, the smaller the 10-63, the better the picture quality. If the picture color is strange or it is split into several parts, it is necessary to increase the value, collect one picture each time to ensure that the collected image quality and fluency meet the requirements, and then ESP32-CAM tries to connect to the specified WIFI network. If the connection is successful, it will obtain its own IP address in the network to prepare for subsequent data transmission. ESP32-CAM will establish a UDP connection with the host computer. The UDP protocol is very suitable for the transmission of real-time image data due to its high efficiency and lightweight characteristics. Through UDP transmission, ESP32-CAM can transmit the collected image data in real time and quickly. The ESP32-CAM enters the loop acquisition mode, and will continuously capture new image data from the camera, and transmit these data to the host computer in real time through the established UDP communication. These data arrive in the form of byte streams, and need to be properly decoded and converted in format for subsequent processing. The server uses C# to write the host computer program, which is responsible for receiving the image data sent by the client, and performing subsequent processing and recognition work. C# has good readability, strong ease of use, and is cross-platform. The host computer program loads the pre-trained YOLOv5n model, and uses the loaded YOLOv5n model to detect and identify underwater dynamic objects in the received image. This process includes image preprocessing, model reasoning and post-processing, and finally outputs the recognition result, which is displayed to the user in a visual form, and the recognized image with annotations is displayed through the PictureBox control. The present invention can transmit the captured underwater image signal to the host computer system written based on C# in real time through the WIFI communication of the camera of ESP32CAM, ensuring the real-time and accuracy of data transmission, and can detect and identify underwater dynamic targets in real time, with good actual use effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1It is the working flow chart of the lower computer of the present invention.

[0012] Figure 2 The following is a C# host computer workflow diagram of the present invention.

[0013] Figure 3 It is a schematic diagram of image reception and analysis of the present invention.

[0014] Figure 4 This is a diagram of the serial port monitoring interface of the present invention.

[0015] Figure 5 This is a diagram of the interface of the NetAssist network debugging tool of the present invention.

[0016] Figure 6 This is the normal working diagram of the host computer of the present invention.

[0017] Figure 7 This is the goldfish detection and identification diagram of the present invention.

[0018] Figure 8 This is the turtle detection and identification diagram of the present invention. DETAILED DESCRIPTION

[0019] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Example

[0020] As attached Figure 1 , Attachment Figure 2 and attached Figure 3 As shown, the present invention provides an underwater dynamic target detection and recognition method based on a C# host computer and ESP32CAM, comprising the following steps: Step 1: First, initialize ESP32-CAM, set key parameters, define the packet size for sending pictures, each image packet is up to 1024 bytes, set the sampled image size and resolution to 800*600, set the quality of the picture, the smaller the value of 10-63, the better the quality of the picture. If the picture color is strange or split into several parts, you need to increase the value. Collect one picture at a time to ensure that the quality and fluency of the collected image meet the requirements; Step 2: ESP32-CAM tries to connect to the specified WIFI network. If the connection is successful, it will obtain its own IP address in the network and prepare for subsequent data transmission; Step 3: ESP32-CAM will establish a UDP connection with the host computer. The UDP protocol is very suitable for the transmission of real-time image data due to its high efficiency and lightweight. Through UDP transmission, ESP32-CAM can send the collected image data to the host computer in real time and quickly; Step 4: ESP32-CAM enters the loop acquisition mode, which continuously captures new image data from the camera and transmits this data to the host computer in real time through the established UDP communication. This data arrives in the form of a byte stream and needs to be properly decoded and format converted for subsequent processing. Step 5: The server uses C# to write a host computer program, which is responsible for receiving the image data sent by the client and performing subsequent processing and recognition work. C# has good readability, strong ease of use, and is cross-platform. The host computer program loads the pre-trained YOLOv5n model and uses the loaded YOLOv5n model to detect and recognize underwater dynamic objects in the received images. This process includes image preprocessing, model reasoning and post-processing, and finally outputs the recognition results; Step 6: Display the recognition results to the user in a visual form by using the PictureBox control to display the annotated recognition image.

[0021] The host computer software C# is mainly responsible for the following contents: (a) Image data reception: responsible for receiving real-time image data stream from the ESP32-CAM device; (b) Image decoding and display: Decode the image data received in the previous step and display the decoded image in real time on the software interface; (c) Object detection processing: The decoded image data is passed to the background YOLOv5n object detection model for processing. This process involves format conversion and preprocessing of the image data to meet the input requirements of the YOLOv5n model. (d) Display and save detection results: Receive the target detection results returned by the YOLOv5n model, mark the detected targets on the original image, and save these results to a local file for subsequent analysis and use.

[0022] The UDP is used to complete the reception and analysis of the wireless communication module on the data packet of the wireless network. The code to implement this function includes: server_client = newSocket(AddressFamily.InterNetwork,SocketType.Dgram,ProtocolType.Udp), create a UDP protocol Socket, use the IPv4 protocol family, datagram type Socket type; server_client.Bind(newIPEndPoint(IPAddress.Parse(this.textBox1.Text),int.Parse(this.textBox2.Text))), binds the Socket to the local IP address and port, and uses the two text boxes (textBox1 and textBox2) in the form to obtain the IP address and port number; RThread = new Thread(ReceiveMsg), creates a thread (RThread) to execute the logic of receiving UDP data packets; EndPoint point = new IPEndPoint(IPAddress.Any, 0), used to receive data from the sender; int length = server_client.ReceiveFrom(bufferCatch, ref point), get socket data; if ((bufferCatch[0] =='o') && (bufferCatch[1] =='k')), when the ok message is received, it means that the next packet will be received immediately, and the last transmitted image information has been received.

[0023] The WiFi module configuration includes: (a) WiFi.mode(WIFI_STA), connect to WiFi, the mode is STA mode; (b) WiFi.begin(ssid,password), connect to the router, ssid is the name of the connected WIFI, password is the password of the connected WIFI; (c) If the WIFI connection is successful, the local IP address will be printed out according to the instructions. If the connection is unsuccessful, a failed connection will appear and a new connection will be established after 1000 milliseconds.

[0024] The UDP configuration includes: (a) if (udp.connect(IPAddress(192,168,43,14),8080)), use the WIFI module to realize the UDP communication connection with the host computer, where 192.168.43.14 is the IP address of the host computer and 8080 is the port number of the host computer; (b) Serial.println("UDP connected"), if UDP is successfully connected, the characters "UDP connected" will be printed out.

[0025] System debugging and analysis: 1. Preparation for debugging We chose to use a water tank to simulate the underwater environment for testing. The specific debugging plan is as follows: (1) Experimental environment construction Prepare the tank: Choose a tank that is large enough to simulate an actual underwater environment.

[0026] Fill water: Fill the water tank with clean water, ensure the water quality is clean to reduce interference with the equipment, and place the purchased goldfish and turtles into the water tank.

[0027] Install the equipment: Install the ESP32-CAM in a suitable location and ensure that the camera can clearly capture the underwater target.

[0028] Connect power and data cables: Provide stable power to the ESP32-CAM and connect the data cables to communicate with the host computer.

[0029] (2) Detecting hardware equipment Check the power supply: Make sure the power supply of ESP32-CAM is connected correctly and can be powered normally.

[0030] Hardware self-test: Start ESP32-CAM and check whether its hardware functions are normal, such as whether the camera is working.

[0031] Connection test: Connect ESP32-CAM and the host computer via a data cable to test whether the communication is normal.

[0032] (3) Communication detection Load the model: Deploy the YOLOv5n model file to C# to ensure that the model can run correctly.

[0033] Functional test: Use actual underwater targets to test whether the targets can be correctly identified and detected.

[0034] Host computer interface debugging: Open the host computer interface to ensure that the images captured by ESP32-CAM can be displayed in real time and the recognition results can be received.

[0035] 2. Debugging tools: Arduino IDE platform + NetAssist network debugging tool +, Visual Studio 2022 host computer programming platform.

[0036] 3. Model training:

[0037] The YOLOv5n training process has the following parts: Divide the dataset: Divide the dataset into proportions (e.g. 80% training set, 10% validation set, 10% test set). Make sure each subset contains enough diversity to train a model with strong generalization ability.

[0038] Model training: Use the deep learning framework PyTorch to train the YOLOv5n model and adjust training parameters such as learning rate, weight, and number of training rounds.

[0039] Model evaluation: During the training process, YOLOv5n will evaluate the performance of the model on the validation set and output evaluation indicators such as mAP (mean Average Precision). After the training is completed, the model is finally evaluated using the test set, and the model's Precision, Recall and other indicators are calculated and output.

[0040] System program debugging

[0041] (1) Image acquisition debugging When debugging ESP32-CAM image acquisition, first, make sure ESP32-CAM is in burning mode, select the serial port and burning configuration to burn. After burning is completed, press the RST button, ESP32-CAM will restart and start executing the program just burned. Figure 4 As shown in the figure, in the serial monitor, "WiFi connected!!!", "UDP connected", "Camera Init OK!!!" are printed out, indicating that the Wi-Fi connection is successful, UDP is connected, and the Camera is turned on.

[0042] After the debugging is successfully executed, open the NetAssist network debugging tool, such as Figure 5 As shown in the figure, in the NetAssist interface, select "UDP" for the communication protocol and enter the IP address and port number. Click to open the listening interface to the specified IP address and port number. Open the host computer and click "Open". The host computer starts to read and display the captured images. Figure 6 As shown, the host computer works normally at this time.

[0043] If problems are found in data reception, such as packet loss, discontinuous reception, or incorrect data content, you may need to check the following aspects: whether the sending settings of the lower computer are correct, including the target IP address and port number; whether the network connection is stable, and whether there is network delay or packet loss; whether the format and content of the data packet are as expected, and whether there are any encoding or parsing errors.

[0044] (2) Dynamic target recognition debugging First, make sure the PC and ESP32-CAM device are connected to the same Wi-Fi hotspot to enable mutual communication.

[0045] Next, start the host computer program written in C#. In the host computer program, you need to fill in the IP address and port number. These parameters are used to specify the network connection information of the ESP32-CAM device so that the host computer can correctly establish a communication connection with it. After filling in the IP address and port number, click the "Start" button. At this point, the host computer program will start listening to the specified port and wait for the data packet sent by ESP32-CAM. Once the data packet is received, the program will integrate the data into a complete picture and display it in the control.

[0046] The program then calls the pre-trained model file to perform target object detection and recognition. During the recognition process, the program analyzes the image and detects the species of the target object. Figure 7 and Figure 8 These are the detection results at different stages. This system can correctly detect goldfish and turtles.

[0047] The present invention first initializes ESP32-CAM, sets key parameters, defines the packet segmentation size of the sent picture, each image packet is 1024 bytes at most, sets the sampled picture size and resolution to 800*600, sets the quality of the picture, the smaller the 10-63, the better the picture quality. If the picture color is strange or it is split into several parts, it is necessary to increase the value, collect one picture each time to ensure that the collected image quality and fluency meet the requirements, and then ESP32-CAM tries to connect to the specified WIFI network. If the connection is successful, it will obtain its own IP address in the network to prepare for subsequent data transmission. ESP32-CAM will establish a UDP connection with the host computer. The UDP protocol is very suitable for the transmission of real-time image data due to its high efficiency and lightweight. Through UDP transmission, ESP32-CAM can send the collected image data to the host computer in real time and quickly. ESP32-CAM enters a cyclic acquisition mode and continuously captures new image data from the camera and transmits it through The established UDP communication transmits these data to the host computer in real time. These data arrive in the form of byte streams and need to be properly decoded and format converted for subsequent processing. The server uses C# to write a host computer program, which is responsible for receiving the image data sent by the client and performing subsequent processing and recognition work. C# has good readability, strong ease of use, and is cross-platform. The host computer program loads a pre-trained YOLOv5n model and uses the loaded YOLOv5n model to detect and recognize underwater dynamic objects in the received image. This process includes image preprocessing, model reasoning and post-processing, and finally outputs the recognition result, which is displayed to the user in a visual form. The recognized image with annotations is displayed through the PictureBox control. The present invention can transmit the captured underwater image signal to the host computer system written based on C# in real time through the WIFI communication of the camera of ESP32CAM, ensuring the real-time and accuracy of data transmission, and can detect and recognize underwater dynamic targets in real time, with good actual use effect.

[0048] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for underwater dynamic target detection and recognition based on C# host computer and ESP32CAM, characterized in that: The following steps are involved: Step 1: First, initialize ESP32-CAM, set key parameters, define the packet size for sending pictures, each image packet is up to 1024 bytes, set the sampled image size and resolution to 800*600, set the quality of the picture, the smaller the value of 10-63, the better the quality of the picture. If the picture color is strange or split into several parts, you need to increase the value. Collect one picture at a time to ensure that the quality and fluency of the collected image meet the requirements; Step 2: ESP32-CAM connects to the specified WIFI network. If the connection is successful, it will obtain its own IP address in the network and prepare for subsequent data transmission; Step 3: ESP32-CAM will establish a UDP connection with the host computer. Through UDP transmission, ESP32-CAM can send the collected image data to the host computer in real time and quickly; Step 4: ESP32-CAM enters the loop acquisition mode, which continuously captures new image data from the camera and transmits this data to the host computer in real time through the established UDP communication. This data arrives in the form of a byte stream and needs to be properly decoded and format converted for subsequent processing; Step 5: The server uses C# to write a host computer program, which is responsible for receiving the image data sent by the client and performing subsequent processing and recognition. The host computer program loads the pre-trained YOLOv5n model and uses the loaded YOLOv5n model to detect and recognize underwater dynamic objects in the received images. This process includes image preprocessing, model inference and post-processing, and finally outputs the recognition results. Step 6: Display the recognition results to the user in a visual form by using the PictureBox control to display the annotated recognition image.

2. The underwater dynamic target detection and recognition method based on C# host computer and ESP32CAM according to claim 1 is characterized in that: The host computer software C# is responsible for the following: (a) Image data reception: responsible for receiving real-time image data stream from the ESP32-CAM device; (b) Image decoding and display: Decode the image data received in the previous step and display the decoded image in real time on the software interface; (c) Object detection processing: The decoded image data is passed to the background YOLOv5n object detection model for processing. This process involves format conversion and preprocessing of the image data to meet the input requirements of the YOLOv5n model. (d) Display and save detection results: Receive the target detection results returned by the YOLOv5n model, mark the detected targets on the original image, and save these results to a local file for subsequent analysis and use.

3. The underwater dynamic target detection and recognition method based on C# host computer and ESP32CAM according to claim 1 is characterized in that: The UDP is used to complete the reception and analysis of the wireless communication module on the data packet of the wireless network. The code to implement this function includes: server_client = newSocket(AddressFamily.InterNetwork,SocketType.Dgram,ProtocolType.Udp), creates a UDP protocol Socket, uses the IPv4 protocol family, and the datagram type of Socket; server_client.Bind(newIPEndPoint(IPAddress.Parse(this.textBox1.Text),int.Parse(this.textBox2.Text))), binds the Socket to the local IP address and port, and uses the two text boxes (textBox1 and textBox2) in the form to obtain the IP address and port number; RThread = new Thread(ReceiveMsg), creates a thread (RThread) to execute the logic of receiving UDP packets; EndPoint point = new IPEndPoint(IPAddress.Any, 0), used to receive data from the sender; int length = server_client.ReceiveFrom(bufferCatch, ref point), get socket data; if ((bufferCatch[0] =='o') && (bufferCatch[1] =='k')), when the ok message is received, it means that the next packet will be received immediately, and the last transmitted image information has been received.

4. The underwater dynamic target detection and recognition method based on C# host computer and ESP32CAM according to claim 1 is characterized in that: The WiFi module configuration includes: (a) WiFi.mode(WIFI_STA), connect to WiFi, the mode is STA mode; (b) WiFi.begin(ssid,password), connect to the router, ssid is the name of the connected WIFI, password is the password of the connected WIFI; (c) If the WIFI connection is successful, the local IP address will be printed out according to the instructions. If the connection is unsuccessful, a failed connection will appear and a new connection will be established after 1000 milliseconds.

5. The underwater dynamic target detection and recognition method based on C# host computer and ESP32CAM according to claim 1 is characterized in that: The UDP configuration includes: (a) if (udp.connect(IPAddress(192,168,43,14),8080)), use the WIFI module to realize the UDP communication connection with the host computer, where 192.168.43.14 is the IP address of the host computer and 8080 is the port number of the host computer; (b) Serial.println("UDP connected"), if UDP is successfully connected, the characters "UDP connected" will be printed out.

Images