An intelligent fish box on a ship

Abstract

This invention discloses a shipborne intelligent fish box, relating to the fields of artificial intelligence and fisheries information technology. It includes a fish box shell, a Jetson embedded main control module, a ZED stereo camera, an electronic scale module, a light source control module, a network communication module, and a cloud data platform. After the user presses the weighing button, the camera acquires RGB and depth images of the fish. The main control module identifies the fish species using the YOLOv11 algorithm, outputs key point coordinates, and calculates the fish length and width using 3D back projection combined with camera intrinsic parameters. The electronic scale simultaneously measures the weight. All data is encapsulated as structured information, displayed on a local screen, and then uploaded to the cloud via the network. The device achieves fully automated integration of fish identification, measurement, weighing, recording, and uploading, adapting to complex shipborne environments, ensuring data consistency and traceability, and is suitable for scenarios such as nearshore catch monitoring, aquaculture sampling, and compliance traceability.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and fisheries information technology, and more specifically to a shipborne intelligent fish box. Background Technology

[0002] Against the backdrop of rapid development in fisheries informatization and intelligent sensing technologies, the demand for integrated and automated fish detection, measurement, and data management in scenarios such as nearshore catch monitoring, scientific sampling for aquaculture, and compliant traceability is becoming increasingly urgent. Currently, in the fisheries sector, fish-related data collection mainly relies on experimental systems spliced ​​together from traditional PCs and peripherals, or solutions loosely combined from single functional modules such as identification, length measurement, and weighing. These solutions are difficult to adapt to the complex environment and high-efficiency requirements of shipboard operations and generally suffer from numerous technical deficiencies.

[0003] The core problems of existing technologies are concentrated in five aspects: First, the links are severely fragmented, with weighing, length measurement, and image recognition functions operating independently. Data cannot be automatically matched and integrated into standardized records at the device end, resulting in cumbersome and costly post-event archiving and traceability processes. Second, the human-computer interaction design lacks engineering adaptation and is not optimized for scenarios such as complex sea conditions and tight time constraints in deck operations. It lacks practical functions such as "connect to the network and use immediately, automatic start-up, and clear operable signals," resulting in poor operability. Third, the measurement accuracy and stability are insufficient. Length measurement is mostly based on two-dimensional pixels or a single sensor, which is easily affected by changes in fish posture, light fluctuations, and surface reflections, resulting in large fluctuations and poor repeatability of measurement results. Fourth, the deployment and maintenance thresholds are high, relying on high-computing-power hosts or complex peripheral connection methods, making it difficult to meet the requirements for long-term stable operation in a shipboard environment. Fifth, the data closed loop is missing. There is a lack of standardized communication protocols for end-cloud collaboration, making it impossible to form a complete link of "result determination - single packet - instant reporting" at the device end, which makes it difficult to meet the requirements of fisheries supervision and production management for data timeliness and reliability.

[0004] To address the aforementioned technical challenges and achieve a fully integrated and automated process for fish identification, measurement, weighing, recording, and uploading in shipboard scenarios, there is an urgent need to develop an intelligent device that deeply integrates hardware and software, adapts to complex maritime operating environments, and features a closed-loop data link. Summary of the Invention

[0005] In view of this, the present invention provides a shipborne intelligent fish box to solve the problems existing in the background art.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: A shipborne intelligent fish box includes: a fish box shell, a Jetson embedded main control module, an image acquisition module, an electronic scale module, a light source control module, a network communication module, and a cloud data platform. The fish box shell has a window, an operation unit, and a display unit. The window serves as a fish inlet for placing the fish to be tested. The operation unit is wired to the Jetson embedded main control module, and the display unit displays the weighing result. The Jetson embedded main control module, image acquisition module, electronic scale module, light source control module, and network communication module are all located inside the fish box shell. When a button on the operation unit is pressed, the Jetson embedded main control module acquires image data from the window through the image acquisition module, performs body length and width identification and fish body recognition, and obtains the recognition result. The electronic scale module measures the weight of the fish to be tested. The light source control module provides supplementary lighting for the image acquisition module. The network communication module sends the weight and recognition result to the cloud data platform.

[0007] Preferably, the image acquisition module uses a ZED stereo camera to acquire RGB and depth images of the fish to be tested, and transmits the RGB and depth images to the Jetson embedded main control module.

[0008] Preferably, the Jetson embedded main control module includes: The fish body recognition unit runs the YOLOv11 algorithm, takes the RGB image as input, and outputs the fish body target box, confidence score, category index and key point coordinates. The fish body target with the highest confidence score is taken as the output result to obtain the target box and key point set. The fish body calculation unit reads the intrinsic parameters f from the ZED stereo camera. x f y c x c y And by using the pixel coordinates and depth values ​​of the key points for 3D back projection, a 3D model of the key points is constructed: ; ; ; Where (u,v) are the pixel coordinates of the key point, and D is the depth at that pixel; The three-dimensional distance between the coordinates of the key point for measuring the length of the mouth and the coordinates of the key point for measuring the center of the caudal fin: ; The width is the three-dimensional distance between the coordinates of the key point at the front end of the upper fin and the coordinates of the key point at the front end of the lower fin. ; in, The coordinates of the key points of the mouth, The coordinates of the key point at the center of the caudal fin are: The coordinates of the key point at the front end of the upper fin are: The coordinates are the key points at the front end of the lower fin.

[0009] Preferably, it also includes a window cover plate disposed on the side end of the window and connected by a hinge.

[0010] Preferably, the display unit is a fish box screen. The Jetson embedded main control module encapsulates the recognition result and the light control quantity of the light source control module into downlink data frames and sends them to the electronic scale module. The electronic scale module then displays the weight of the fish to be measured and the downlink data frames on the fish box screen.

[0011] As can be seen from the above technical solution, compared with the prior art, the present invention discloses a shipborne intelligent fish box that achieves highly robust target detection and key point measurement on an embedded end (Jetson orin NX). Combined with electronic scale reading and network communication, it completes a one-time structured data packet (fish body, body length, body width, weight, timestamp, image evidence), and automatically starts the operation after successful network connection, providing a status prompt that detection can start through supplementary lighting / light indicators. In terms of operation, the measurement event is atomically triggered by the front panel buttons, ensuring the consistency, traceability and upload success rate of the records. In terms of system, it realizes a closed loop between the end and the cloud, which can display the results locally and push the data to the backend platform in real time for statistical analysis, law enforcement evidence collection, fishery resource monitoring and production management. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0013] Figure 1 The system flowchart provided for this invention; Figure 2 The front view of the smart fish box provided by the present invention; Figure 3 This is a top view of the smart fish box provided by the present invention; In the diagram, 1 is the light source strip; 2 is the Jetson Orin NX; 3 is the switch; 4 is the weight sensor; 5 is the PCB development board; 6 is the battery pack; 7 is the transformer; 8 is the ZED binocular camera; 9 is the charging port; 10 is the fish release port; 11 is the indicator light; 12 is the weighing button; 13 is the power switch; 14 is the zeroing button; 15 is the serial port screen; and 16 is the network port. Detailed Implementation

[0014] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0015] This invention discloses a shipborne intelligent fish box, comprising: a fish box shell, a Jetson embedded main control module, an image acquisition module, an electronic scale module (MCU terminal), a light source control module, a network communication module, and a cloud data platform. The fish box shell has a window, an operation unit, and a display unit. The window serves as a fish inlet for placing the fish to be tested. The operation unit is wired to the Jetson embedded main control module, and the display unit displays the weighing result. The Jetson embedded main control module, the image acquisition module, the electronic scale module, the light source control module, and the network communication module are all housed within the fish box shell. Internally, after pressing the button on the operation unit, the Jetson embedded main control module acquires image data from the window through the image acquisition module, performs body length and width identification and fish body recognition, and obtains the recognition result; the electronic scale module measures the weight of the fish and communicates with Jetson via the Socket protocol; the light source control module automatically turns on or off according to system instructions to provide supplemental lighting for the image acquisition module; the network communication module connects to the switch via Ethernet to realize bidirectional communication between the MCU and Jetson, sending the weight and recognition result to the cloud data platform to record the fish detection result.

[0016] Preferably, the image acquisition module uses a ZED stereo camera to acquire RGB and depth images of the fish to be tested, and transmits the RGB and depth images to the Jetson embedded main control module.

[0017] Preferably, the Jetson embedded main control module includes: The fish body recognition unit runs the YOLOv11 algorithm, takes the RGB image as input, and outputs the fish body target box, confidence score, category index and key point coordinates. The fish body target with the highest confidence score is taken as the output result to obtain the target box and key point set. The fish body calculation unit reads the intrinsic parameters f from the ZED stereo camera. x f y c x c y And by using the pixel coordinates and depth values ​​of the key points for 3D back projection, a 3D model of the key points is constructed: ; ; ; Where (u,v) are the pixel coordinates of the key point, and D is the depth at that pixel; The three-dimensional distance between the coordinates of the key point for measuring the length of the mouth and the coordinates of the key point for measuring the center of the caudal fin: ; The width is the three-dimensional distance between the coordinates of the key point at the front end of the upper fin and the coordinates of the key point at the front end of the lower fin. ; in, The coordinates of the key points of the mouth, The coordinates of the key point at the center of the caudal fin are: The coordinates of the key point at the front end of the upper fin are: The coordinates are the key points at the front end of the lower fin.

[0018] Preferably, it also includes a window cover plate disposed on the side end of the window and connected by a hinge.

[0019] Preferably, the display unit is a fish box screen. The Jetson embedded main control module encapsulates the recognition result and the light control quantity of the light source control module into downlink data frames and sends them to the electronic scale module. The electronic scale module then displays the weight of the fish to be measured and the downlink data frames on the fish box screen.

[0020] The algorithm and system operation process are as follows: Figure 1 As shown: Step S1: Power-on initialization and network self-test.

[0021] After the device is powered on, Jetson starts system services and initializes the camera, communication link, and GPIO buttons; then it performs a network connectivity test. When the network is available, the fish detection script is automatically started, and a light control command is sent to the MCU to illuminate the internal lighting of the fish box, indicating that it has entered a detectable state (the light control field is 0x01 to indicate that it is lit). Jetson and the MCU establish a connection via Ethernet socket, with the MCU acting as the server, IP address 192.168.1.7, port 8887, and using a non-acknowledgment communication mechanism to complete data exchange.

[0022] Step S2: Button trigger and data synchronous acquisition.

[0023] When the user places the fish into the fish box and presses the "Weigh" button on the front of the device, the button signal triggers an interrupt via Jetson GPIO, and the system enters a complete data acquisition process: Jetson immediately captures a frame of RGB image from the ZED camera and simultaneously obtains the corresponding depth map (depth unit is mm). At the same time, the MCU completes the weighing and status acquisition, and sends the weighing data frame to Jetson. Jetson reads the data frame through the Socket for this recording.

[0024] Step S3: Weighing data frame parsing (MCU→Jetson).

[0025] The data frame structure sent by the MCU is: AA [weight H] [weight L] [temperature] [battery] FF.

[0026] Jetson parses according to the following rules: Weight (g): Weight=WeightH×256+WeightL; Temperature (°C): Temp = [temperature]; Battery power (%): Power = [Battery power].

[0027] The above analysis results are bound one-to-one with the captured images for subsequent display and uploading.

[0028] Step S4: Edge-side visual reasoning and keypoint / category output (YOLO).

[0029] Jetson performs edge-side inference on captured RGB images, outputting the fish bounding box, confidence score, species ID, and keypoint coordinates. Keypoints are defined as follows: 0—Mouth; 1—Tip of upper fin; 2—Tip of lower fin; 3—Middle section of upper fin; 4—Middle section of lower fin; 5—Tip of upper fin; 6—Tip of lower fin; 7—Center of caudal fin.

[0030] The system selects the fish target with the highest confidence level as the detection object and obtains the target bounding box and key point set.

[0031] Step S5: Calculate the length / width of the fish (based on depth and camera intrinsics).

[0032] The system reads camera intrinsic parameters f in real time from ZED. x f y c x c y And by using the pixel coordinates and depth values ​​of key points to perform 3D back projection, the 3D coordinates of the key points are constructed:

[0033] Where (u,v) are the pixel coordinates of the key point, and D is the depth (mm) at that pixel.

[0034] Then calculate: Single length: The three-dimensional distance between key point 0 (mouth) and key point 7 (center of caudal fin)

[0035] Single line width: the three-dimensional distance between key point 1 (front end of upper fin) and key point 2 (front end of lower fin).

[0036] The obtained length / width results are directly expressed in mm.

[0037] Step S6: Result feedback and display on screen (Jetson→MCU).

[0038] Jetson encapsulates the length, width, category index, and lighting control parameters into a downlink data frame and sends it to the MCU. The frame structure is as follows: FF [Length H] [Length L] [Width H] [Width L] [Type] [Light] AA.

[0039] in: Length (mm): Length = LenH × 256 + LenL; Width (mm): Width = WidH × 256 + WidL; The category field is a category index (species id); The light field 0x01 indicates that the light is on, and 0x00 indicates that the light is off.

[0040] After receiving the data, the MCU displays information such as length, weight, and category on the fish box screen, and provides interactive prompts in conjunction with the light status.

[0041] Step S7: Upload structured data packets to the local cache queue.

[0042] Jetson encapsulates fields such as "fish box number / region / fish species name / single length / single weight / test result / catch status / test photo / test time / data source / remarks" along with image evidence into structured data, which is then uploaded to the backend website via an HTTP interface. At the same time, a local CSV cache queue is set up to ensure data traceability and replayability.

[0043] The category index (species ID) is automatically detected by the YOLO vision model running on the Jetson edge during the inference phase, and its generation mechanism is as follows: Step C1: Input and Preprocessing.

[0044] After a button is pressed, Jetson captures a frame of RGB image, scales and normalizes it according to the model input size, and then sends the resulting tensor input to the YOLO network.

[0045] Step C2: The model performs forward inference to obtain candidate targets.

[0046] The YOLO network outputs a set of candidate targets for each input image, where each candidate target contains at least: 1. Target bounding box

[0047] 2. Confidence level

[0048] 3. Category score vector (K is the number of fish species); 4. Key point coordinate set

[0049] Step C3: Calculate the category index (Top-1 classification).

[0050] For each candidate target, the system uses softmax to obtain the class probability from the class score vector:

[0051] The category index takes the category with the highest probability:

[0052] The system selects target i with the highest confidence level as the final detection object:

[0053] The final category index is:

[0054] The species_id is the source of the [type] field in the downlink frame of the communication protocol.

[0055] Step C4: Category name mapping and display / upload field generation.

[0056] The system has a built-in "Category Index - Fish Species Name" mapping table (JSON file). The Chinese / Latin name of the fish species can be retrieved based on the species_id, for use in: Display the "name of the catch" on the screen; Fill in fields such as "Catch Name / Test Result" in the uploaded data packet; In the communication frame, species_id is still used as a compact field to send back to the MCU, ensuring stable transmission and minimizing bandwidth usage.

[0057] Step C5: Coupling relationship with key points and length measurement.

[0058] The coordinates of key points for the same target are synchronously output by YOLO. Based on this, the system extracts depth from the depth map and performs 3D back projection to calculate Length / Width. That is, the category index and key points both come from the same edge inference, ensuring that "category-length-weight-image evidence" strictly correspond within the same button trigger event, avoiding data mismatch.

[0059] like Figures 2 to 3 As shown, the present invention provides a shipborne intelligent fish box, including a fish box shell and various functional components integrated inside and on the surface of the shell.

[0060] The system includes: a fish inlet 10 (window); an indicator light 11, a weighing button 12, a power switch 13, and a zeroing button 14, which together form the operating unit; a serial port screen 15 (display unit); a network communication module including a network port 16 and a switch 3; a light source control module consisting of a light strip 1; a Jetson embedded main control module consisting of a Jetson orin NX 2; a weight sensor 4 (electronic scale module); and a ZED binocular camera 8 (image acquisition module).

[0061] In terms of structural layout, the top of the fish box shell has a fish-holding opening 10 for placing the fish to be tested. A ZED binocular camera 8 is installed directly below the inside of the fish-holding opening 10 to collect visual data of the fish. A light strip 1 surrounds the inside of the fish-holding opening 10, with adjustable brightness to provide a stable and diverse lighting environment. A weight sensor 4 is installed at the bottom inside the fish box shell to sense the weight of the fish in real time. The operation panel of the fish box shell includes indicator lights 11, a weighing button 12, a power switch 13, a zeroing button 14, and a serial port display screen 15 for easy user interaction and result viewing.

[0062] In terms of electrical connections and control logic, the Jetson Orin NX 2 serves as the core computing unit, establishing data connections with the image acquisition module 8, display unit 15, switch 3, and PCB development board 5. Specifically, PCB development board 5 is responsible for acquiring analog signals from the weight sensor 4 and converting them into digital signals, while also responding to the trigger signals from the weighing button 12 and the zeroing button 14. Switch 3 connects to the external network via Ethernet port 16 and is responsible for local area network communication between the main control module and its internal components. For system power supply, charging port 9 connects to transformer 7, providing stable power support to all electronic components through battery pack 6.

[0063] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. The methods disclosed in the embodiments are described simply because they correspond to the methods disclosed in the embodiments; relevant parts can be found in the method section.

[0064] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A shipboard smart fish box, characterized in that, The application relates to a fish box shell, a Jetson embedded host module, an image acquisition module, an electronic scale module, a light source control module, a network communication module and a cloud data platform, wherein the fish box shell is provided with a window, an operation unit and a display unit; the window is used as a fish placing opening for placing fish to be measured; the operation unit is connected with the Jetson embedded host module in circuit; and the display unit displays the weighing result; the Jetson embedded host module, the image acquisition module, the electronic scale module, the light source control module and the network communication module are arranged in the fish box shell; after a button of the operation unit is pressed, the Jetson embedded host module obtains image data at the window through the image acquisition module, carries out fish body length and width measurement and fish body recognition on the fish to be measured, and obtains a recognition result; the electronic scale module measures the weight of the fish to be measured; the light source control module is used for light compensation of the image acquisition module; and the network communication module sends the weight and the recognition result to the cloud data platform. The image acquisition module applies a ZED stereo camera to collect an RGB image and a depth image of the fish to be measured, and transmits the RGB image and the depth image to the Jetson embedded host module.

2. The onboard smart fish box of claim 1, wherein, The Jetson embedded host module comprises:

3. The onboard smart fish box of claim 2, wherein, a fish body recognition unit which runs a YOLOv11 algorithm, takes the RGB image as input, outputs a fish body target frame, a confidence, a category index and key point coordinates, takes the fish body target with the highest confidence as output result, obtains a target frame and a key point set, and outputs the target frame and the key point set. Wherein (u, v) is a key point pixel coordinate, and D is the depth at the pixel. The fish body computing unit reads the intrinsic parameters f of the ZED stereo camera x 、 y 、 x 、 y , and uses the key point pixel coordinates and the depth value to perform three-dimensional back projection to construct the key point three-dimensional: ； ； ； The length is the three-dimensional distance between the key point coordinates of the mouth and the key point coordinates of the center of the tail fin; The width is the three-dimensional distance between the key point coordinates of the front end of the upper fish fin and the key point coordinates of the front end of the lower fish fin; ； The application further comprises a window cover plate which is arranged at the side end of the window and is connected through a hinge. ； wherein, is a mouth key point coordinate, is a tail fin center key point coordinate, is an upper fish fin tip key point coordinate, is a lower fish fin tip key point coordinate.

4. The smart fish box of claim 1, wherein, The display unit is a fish box screen; the Jetson embedded host module encapsulates the recognition result and the light control amount of the light source control module into a downlink data frame and sends the downlink data frame to the electronic scale module; and the electronic scale module displays the weight of the fish to be measured and the downlink data frame on the fish box screen.

5. The smart fish box of claim 1, wherein, ​

Images