Deep sea fish school detection method, device and equipment and storage medium

Abstract

The invention discloses a deep and far sea fish school detection method, device and equipment and a storage medium, and relates to the technical field of smart fishery, and the method comprises the steps: obtaining a preset video stream address of a camera; according to the preset video stream address, a preset detection model is adopted to carry out fish school picture detection, a detection result is obtained, and the preset detection model comprises a depth separable convolution and a lightweight activation function. The intelligent level of deep sea fish school detection can be improved, and the death rate of fishes and the manpower and material resource cost of farmers are reduced.

Description

Technical Field

[0001] The present application relates to the field of smart fishery technology, and in particular to deep-sea and deep-sea fish detection methods, devices, equipment and storage media. Background Art

[0002] With the in-depth development of the field of smart fisheries, it is necessary to monitor and identify underwater fish schools in real time. Object detection is one of the key technologies in the field of computer vision and is widely used in video surveillance, autonomous driving and other fields. Underwater fish school detection is an important technical branch. Although existing algorithms have made significant progress in accuracy, they often face the problem of limited computing resources when deployed on edge devices. At the same time, in real production environments, deep-sea fish farming platforms have harsh natural environments, unstable power supply, and difficult maintenance, which cannot provide a good deployment environment for edge devices.

[0003] Therefore, using a remote server with more computing power to deploy a real-time fish detection algorithm model has become a better solution. When deployed on a remote server, problems such as network latency, data transmission efficiency, and server resource management are often encountered. Among them, network latency and data transmission efficiency will cause data loss, and server resource management will cause data blocking. These problems will affect the effectiveness, immediacy, and image integrity of the fish detection algorithm when calculating the remote underwater camera image in real time. Summary of the invention

[0004] The main purpose of this application is to provide a method, device, equipment and storage medium for deep-sea fish detection, aiming to solve the technical problem of how to improve the intelligence level of deep-sea fish detection and reduce fish mortality and the manpower and material costs of farmers.

[0005] To achieve the above objectives, the present application proposes a method for detecting deep-sea fish schools, the method comprising:

[0006] Get the preset video stream address of the camera;

[0007] According to the preset video stream address, a preset detection model is used to perform fish school image detection to obtain a detection result, wherein the preset detection model includes a depthwise separable convolution and a lightweight activation function.

[0008] In one embodiment, the step of performing fish school image detection using a preset detection model according to the preset video stream address to obtain a detection result includes:

[0009] Upon receiving a start command from the control port, obtaining a real-time fish school image according to the preset video stream address;

[0010] Inputting the real-time fish school image into a preset detection model;

[0011] Based on the preset detection model, the real-time fish school image is processed by deep convolution and the lightweight activation function to obtain a deep convolution output;

[0012] Based on the preset detection model, the deep convolution output is processed by point-by-point convolution and the lightweight activation function to obtain a detection result.

[0013] In one embodiment, the step of processing the real-time fish school image based on the preset detection model through deep convolution and the lightweight activation function to obtain a deep convolution output includes:

[0014] Obtaining an input channel for the real-time fish school image;

[0015] Allocating an independent first convolution kernel to the input channel through the preset detection model;

[0016] Calculating the dot product of the first convolution kernel and the local area of ​​the real-time fish school image to obtain a two-dimensional feature map;

[0017] The two-dimensional feature map is processed by the lightweight activation function to obtain a deep convolution output.

[0018] In one embodiment, the step of processing the deep convolution output by point-by-point convolution and the lightweight activation function based on the preset detection model to obtain the detection result includes:

[0019] Configure a second convolution kernel for the deep convolution output through the preset detection model;

[0020] Performing point-by-point multiplication and accumulation on all channels of the depth convolution output through a second convolution kernel to obtain a convolution feature map;

[0021] The convolution feature map is processed by the lightweight activation function to obtain a detection result.

[0022] In one embodiment, the step of obtaining the preset video stream address of the camera includes:

[0023] Receive push data from the control port;

[0024] The video stream address of the camera is obtained from the push data, and the video stream address is selected by the control port according to the distribution instruction.

[0025] In one embodiment, the step of performing fish school image detection using a preset detection model according to the preset video stream address and obtaining the detection result comprises:

[0026] Acquire real-time messaging protocol video stream data and identification data in the detection result;

[0027] Converting the real-time message transmission protocol video stream data into streaming media data to obtain real-time streaming transmission protocol video stream data;

[0028] The real-time streaming protocol video stream data and the identification data are connected to a front-end device for display.

[0029] In one embodiment, the step of performing fish school image detection using a preset detection model according to the preset video stream address and obtaining the detection result further includes:

[0030] Obtaining identification data in the detection result;

[0031] The identification data is associated with a device identification number and a timestamp and stored.

[0032] In addition, to achieve the above-mentioned purpose, the present application also proposes a deep-sea fish school detection device, the device comprising: a receiving module, used to obtain a preset video stream address of a camera;

[0033] The processing module is used to perform fish school image detection according to the preset video stream address using a preset detection model to obtain a detection result, wherein the preset detection model includes a depth-separable convolution and a lightweight activation function.

[0034] In addition, to achieve the above-mentioned purpose, the present application also proposes a deep-sea fish detection device, which includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the deep-sea fish detection method as described above.

[0035] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the deep-sea fish detection method as described above are implemented.

[0036] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the steps of the deep-sea fish detection method as described above.

[0037] One or more technical solutions proposed in this application have at least the following technical effects:

[0038] By adopting a network model including deep separable convolution and lightweight activation function for fish school image detection, the difficulties faced by fish school detection, such as limited computing resources, network delay, data transmission efficiency and server resource management, which in turn affect the technical problems of the effectiveness, immediacy and image integrity of the fish school detection algorithm when calculating the remote underwater camera image in real time. Compared with the existing technology, the intelligence level of deep-sea fish school detection is improved, and the mortality rate of fish and the manpower and material costs of farmers are reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0040] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0041] Figure 1 A schematic diagram of a process flow provided for the first embodiment of the deep-sea fish school detection method of the present application;

[0042] Figure 2 A schematic diagram of a flow chart provided for the second embodiment of the deep-sea fish school detection method of the present application;

[0043] Figure 3 A flow chart of the third embodiment of the deep-sea fish detection method of the present application;

[0044] Figure 4 A schematic diagram of a process flow provided for Embodiment 4 of the deep-sea fish school detection method of the present application;

[0045] Figure 5 A schematic diagram of a process flow provided for Embodiment 5 of the deep-sea fish school detection method of the present application;

[0046] Figure 6 This is a schematic diagram of the module structure of the deep-sea fish school detection device according to an embodiment of the present application;

[0047] Figure 7 Schematic diagram of the device structure of the hardware operating environment involved in the deep-sea fish detection method in the embodiment of the present application.

[0048] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0049] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0050] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0051] The main solution of the embodiment of the present application is: obtain the preset video stream address of the camera; according to the preset video stream address, use the preset detection model to perform fish school image detection to obtain the detection result, and the preset detection model includes a deep separable convolution and a lightweight activation function.

[0052] In this embodiment, for ease of description, the following description is made with the remote target detection server as the execution entity.

[0053] Because fish school detection in existing technologies faces difficulties in limited computing resources, network delays, data transmission efficiency and server resource management, which in turn affects the effectiveness, immediacy and image integrity of fish school detection algorithms when calculating images from remote underwater cameras in real time.

[0054] The present application provides a solution, which uses a network model including deep separable convolution and lightweight activation function to detect fish school images, thereby improving the intelligence level of deep-sea fish school detection and reducing fish mortality and the manpower and material costs of fish farmers.

[0055] It can be seen from the above embodiments that the present application adopts a network model including deep separable convolution and lightweight activation function for fish school image detection, thereby solving the difficulties faced by fish school detection in limited computing resources, network delays, data transmission efficiency and server resource management, which in turn affects the technical problems of the effectiveness, immediacy and image integrity of the fish school detection algorithm in real-time calculation of the remote underwater camera image, thereby improving the intelligence level of deep-sea fish school detection and reducing the fish mortality rate and the manpower and material costs of farmers.

[0056] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, etc. The following takes a remote target detection server as an example to illustrate this embodiment and the following embodiments.

[0057] Based on this, the present application embodiment provides a method for detecting deep-sea fish schools, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the deep-sea fish detection method of the present application.

[0058] In this embodiment, the deep-sea fish school detection method includes steps S10 to S20:

[0059] Step S10, obtaining the preset video stream address of the camera.

[0060] It should be noted that the camera is a camera placed underwater for obtaining real-time images of fish schools; the video stream address refers to a real-time or on-demand stream of a video content, and this address can be used to transmit video data in the network.

[0061] The video data of the underwater camera at the far sea end is managed and distributed by the management platform after being streamed, and the remote target detection server receives the camera video stream address distributed by the management platform.

[0062] Step S20, according to the preset video stream address, using a preset detection model to perform fish school scene detection to obtain a detection result, wherein the preset detection model includes a depthwise separable convolution and a lightweight activation function.

[0063] It should be noted that the preset detection model is a lightweight target detection model; deep separable convolution is a convolution operation used to build a lightweight network model in deep learning. By splitting the correlation between spatial dimensions and channel dimensions, the number of parameters and the amount of calculation required for convolution calculation are reduced, including deep convolution and point-by-point convolution. Compared with traditional convolution, deep separable convolution significantly reduces the number of parameters and the amount of calculation; lightweight activation functions refer to those in deep learning models, especially in lightweight neural networks, which are used to reduce computational complexity and the number of parameters, including Swish activation function and Mish activation function.

[0064] The present embodiment provides a method for detecting deep-sea fish schools. By adopting a network model including a deep separable convolution and a lightweight activation function to detect fish school images, the difficulties faced by fish school detection, such as limited computing resources, network delays, data transmission efficiency, and server resource management, are solved, thereby solving the technical problems that affect the effectiveness, immediacy, and image integrity of the fish school detection algorithm when calculating the images of a remote underwater camera in real time. This improves the intelligence level of deep-sea fish school detection and reduces the mortality rate of fish and the manpower and material costs of farmers.

[0065] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 2 , step S20 includes steps S21 to S24:

[0066] Step S21, when receiving the start instruction of the control port, obtaining the real-time fish school picture according to the preset video stream address.

[0067] After receiving the start-up command from the control port, the remote target detection server starts working and obtains the real-time underwater fish images captured by the corresponding camera according to the video stream address. It controls the start and stop of the model through the control distribution platform to achieve selective input and output. This eliminates the need for the server to process a large number of camera images at the same time, greatly reducing the computing power pressure of the server's real-time reasoning.

[0068] Step S22, inputting the real-time fish school image into a preset detection model.

[0069] The real-time fish school images are input into the preset detection model in the form of feature maps, and analyzed and processed by the model to achieve fish school detection. In order to reduce the complexity of the model, the simplified target detection network usually reduces the number of convolutional layers and the number of filters in each layer, which helps to reduce the number of model parameters and computing requirements.

[0070] Step S23, based on the preset detection model, the real-time fish school image is processed by deep convolution and the lightweight activation function to obtain a deep convolution output.

[0071] It should be noted that Depthwise Convolution is a key step in depth-separable convolution, which specifically processes each channel of the input feature map without mixing features of different channels together.

[0072] Through the preset detection model, deep convolution is performed on the real-time fish school image. The deep convolution adopts a lightweight activation function, and the model output after convolution is the deep convolution output.

[0073] In a feasible implementation, step S23 may include steps S231 to S234:

[0074] Step S231, obtaining the input channel of the real-time fish school image.

[0075] It should be noted that the input channel (in_channels) refers to the number of channels of input data received by the convolutional layer in the convolutional neural network.

[0076] Step S232: assigning an independent first convolution kernel to the input channel through the preset detection model.

[0077] It should be noted that the first convolution kernel is a convolution kernel of size K×K defined by the detection model according to the characteristics of the input data and the requirements of the receptive field during the deep convolution process.

[0078] In the specific implementation, a convolution kernel is assigned to each input channel of the feature map. For example, assuming the input size of the feature map is H×W×C in, where H is the height, W is the width, C in Is the number of channels input channel is C in , then there will be C in There are such convolution kernels, each of which acts independently on a channel.

[0079] Step S233, calculating the dot product of the first convolution kernel and the local area of ​​the real-time fish school image to obtain a two-dimensional feature map.

[0080] For each input channel, the corresponding K×K convolution kernel slides over the entire feature map, that is, slides in the H×W space. At each position, the dot product of the convolution kernel and the local area of ​​the feature map is calculated to obtain a single value. All these values ​​are combined to form a new two-dimensional feature map. The size of this new feature map is still H×W (unless padding or stride is applied).

[0081] Step S234: Process the two-dimensional feature map using the lightweight activation function to obtain a deep convolution output.

[0082] A nonlinear activation function is applied to the convolution result of each channel to introduce nonlinearity and help the network learn more complex features. The lightweight activation functions Swish and Mish are used. These activation functions are more efficient than traditional ReLU in calculation. After each input channel undergoes deep convolution, a corresponding output feature map will be generated. If the input has C in channels, the output will also have C in channels, and the feature map size of each channel is H×W.

[0083] By performing deep convolution on the real-time fish school images, preliminary processing of the fish school images can be achieved.

[0084] Step S24: Based on the preset detection model, the deep convolution output is processed by point-by-point convolution and the lightweight activation function to obtain a detection result.

[0085] It should be noted that pointwise convolution is a technique in convolutional neural network (CNN), which mainly involves convolution operations using a 1x1 convolution kernel.

[0086] In a feasible implementation, step S24 may include steps S241 to S243:

[0087] Step S241: configure a second convolution kernel for the deep convolution output through the preset detection model.

[0088] It should be noted that the second convolution kernel is defined by the detection model according to the characteristics of the input data during the point-by-point convolution process. out ×C out The convolution kernel size, C out It is the number of output channels of the feature map; the output channel (out_channels) refers to the number of channels of the convolutional layer output data in the convolutional neural network.

[0089] Definition C out There are C 1x1 convolution kernels, each of which corresponds to an output channel. out convolution kernels, each with C in A weight.

[0090] Step S242, multiplying and accumulating all channels of the depth convolution output point by point through the second convolution kernel to obtain a convolution feature map.

[0091] For each output channel, use its corresponding 1x1 convolution kernel to perform point-by-point multiplication and accumulation on all channels of the input feature map. Specifically, for each position (i, j) of the input feature map, the calculation is as follows:

[0092]

[0093] Where k is the index of the output channel and m is the index of the input channel.

[0094] Step S243: Process the convolution feature map using the lightweight activation function to obtain a detection result.

[0095] Nonlinear activation functions such as Swish and Mish are applied to the results of point-by-point convolution to increase the nonlinear expression ability of the model. The output of point-by-point convolution is a new feature map with a size of H×W×C out , each output channel is a weighted sum of the input channels, and the weight is determined by the parameters of the 1x1 convolution kernel.

[0096] By performing point-by-point convolution on the real-time fish school images, the amount of model calculation and parameters can be reduced.

[0097] The present embodiment provides a method for detecting deep-sea fish schools. When a start-up instruction of a control port is received, a real-time fish school image is obtained according to the preset video stream address; the real-time fish school image is input into a preset detection model; based on the preset detection model, the real-time fish school image is processed by deep convolution and the lightweight activation function to obtain a deep convolution output; based on the preset detection model, the deep convolution output is processed by point-by-point convolution and the lightweight activation function to obtain a detection result, thereby reducing the amount of model calculation and the amount of parameters, and reducing the complexity of the model.

[0098] Based on the first embodiment of the present application, in the third embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 3 , step S10 includes steps S11 to S12:

[0099] Step S11, receiving push data from the control port.

[0100] It should be noted that the control port is a platform for distributing video stream addresses and controlling the start and stop of the model.

[0101] The control port manages and distributes the video data of the underwater camera at the far sea end in streaming media, forming push data that is sent to the remote target detection server and received by the remote target detection server.

[0102] Step S12, obtaining the video stream address of the camera from the pushed data, wherein the video stream address is selected by the control port according to the distribution instruction.

[0103] The video stream address of a single camera selected by the control port according to the distribution instruction is obtained from the pushed data, without the need for the server to process a large number of camera images at the same time.

[0104] This embodiment provides a deep-sea fish detection method, which receives push data from a control port; obtains a video stream address of a camera from the push data, and the video stream address is selected by the control port according to a distribution instruction, thereby releasing the computing pressure of real-time reasoning of a server.

[0105] Based on the first embodiment of the present application, in the fourth embodiment of the present application, the same or similar contents as those in the first embodiment can be referred to the above description, and will not be described in detail later. Figure 4 , after step S20, further steps S201 to S203 are included:

[0106] Step S201, obtaining real-time messaging protocol video stream data and identification data in the detection result.

[0107] It should be noted that the real-time messaging protocol video stream data is the real-time messaging protocol (RTMP) video stream data refers to the video data transmitted using the RTMP protocol. RTMP (Real-Time Messaging Protocol) is a real-time messaging protocol developed by Adobe, mainly used for the transmission of streaming media data, based on TCP transmission, with the characteristics of low latency and high reliability; the identification data is the data obtained by intuitive identification after processing the fish school image, including quantity statistics, movement speed estimation and the size and number of fish.

[0108] Step S202: convert the real-time messaging protocol video stream data into streaming media data to obtain real-time streaming protocol video stream data.

[0109] It should be noted that real-time streaming protocol video stream data is video data transmitted using real-time streaming protocols (such as RTSP). These protocols define the format, method and control mechanism of data transmission to ensure that video data can be accurately and efficiently transmitted to the target terminal and decoded and played.

[0110] Real-time message transmission protocol video stream data cannot be directly compatible with most front-end protocols. After setting up the streaming server, streaming media data conversion is performed to output the real-time streaming transmission protocol video stream data, transform the model output into the format of video stream, and cooperate with the streaming server at the same time to transfer data in the format of video stream throughout the process. This is not only more convenient for compatibility with end-to-end application equipment, but also convenient for deep-sea remote video data transmission.

[0111] Step S203: connect the real-time streaming protocol video stream data and the identification data to a front-end device for display.

[0112] It should be noted that the front-end device is a device or component that directly interacts with the user or directly receives user input in the network architecture, communication system or software application, including web, app, etc.

[0113] The present embodiment provides a method for detecting fish schools in the deep sea, obtaining real-time message transmission protocol video stream data and identification data in the detection result; performing streaming media data conversion on the real-time message transmission protocol video stream data to obtain real-time streaming transmission protocol video stream data; connecting the real-time streaming transmission protocol video stream data and the identification data to a front-end device for display, thereby improving the deep-sea remote video data transmission capability.

[0114] Based on the first embodiment of the present application, in the fourth embodiment of the present application, the same or similar contents as those in the first embodiment can be referred to the above description, and will not be described in detail later. Figure 5 , after step S20, steps S01 to S02 are also included:

[0115] Step S01, obtaining identification data in the detection result.

[0116] Obtain identification data such as number statistics, speed estimation, and size of fish in the detection results output by the model.

[0117] Step S02: Associating and storing the identification data with the device identification number and the timestamp.

[0118] It should be noted that the device identification number, also known as the device ID, is a serial number or code used to uniquely identify a specific device; the device timestamp usually refers to the date and time stamp associated with a specific event or data point, which can be used to identify the precise moment when data is generated or an event occurs, and is crucial for operations such as logging, synchronization, sorting and analysis.

[0119] The detection model not only outputs video streams, but also stores the identified data in association with the device ID and timestamp. The stored data can be used to mine and study deeper data associations.

[0120] This embodiment provides a method for detecting deep-sea fish schools, obtaining identification data in the detection results; associating and storing the identification data with a device identification number and a timestamp, and providing a data basis for mining and studying deeper data association relationships.

[0121] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the deep-sea fish detection method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0122] This application also provides a deep-sea fish school detection device, please refer to Figure 6 , the device comprises:

[0123] The receiving module 10 is used to obtain the preset video stream address of the camera.

[0124] The processing module 20 is used to perform fish school image detection according to the preset video stream address using a preset detection model to obtain a detection result, wherein the preset detection model includes a depthwise separable convolution and a lightweight activation function.

[0125] In one embodiment, the processing module 20 is also used to obtain a real-time fish school image according to the preset video stream address when a startup instruction from the control port is received; input the real-time fish school image into a preset detection model; based on the preset detection model, process the real-time fish school image through deep convolution and the lightweight activation function to obtain a deep convolution output; based on the preset detection model, process the deep convolution output through point-by-point convolution and the lightweight activation function to obtain a detection result.

[0126] In one embodiment, the processing module 20 is also used to obtain the input channel of the real-time fish school image; allocate an independent first convolution kernel to the input channel through the preset detection model; calculate the dot product of the first convolution kernel and the local area of ​​the real-time fish school image to obtain a two-dimensional feature map; process the two-dimensional feature map through the lightweight activation function to obtain a deep convolution output.

[0127] In one embodiment, the processing module 20 is also used to configure a second convolution kernel for the deep convolution output through the preset detection model; perform point-by-point multiplication and accumulation on all channels of the deep convolution output through the second convolution kernel to obtain a convolution feature map; and process the convolution feature map through the lightweight activation function to obtain a detection result.

[0128] In one embodiment, the receiving module 10 is further used to receive push data from the control port; obtain the video stream address of the camera from the push data, and the video stream address is selected by the control port according to the distribution instruction.

[0129] In one embodiment, the processing module 20 is further used to obtain the real-time messaging protocol video stream data and the identification data in the detection result; convert the real-time messaging protocol video stream data into streaming media data to obtain real-time streaming protocol video stream data; and connect the real-time streaming protocol video stream data and the identification data to a front-end device for display.

[0130] In one embodiment, the processing module 20 is further used to obtain identification data in the detection result; associate the identification data with the device identification number and the timestamp and store them.

[0131] The deep-sea fish detection device provided by the present application adopts the deep-sea fish detection method in the above-mentioned embodiment, which can solve the technical problem of how to improve the intelligence level of deep-sea fish detection and reduce the mortality rate of fish and the manpower and material costs of farmers. Compared with the prior art, the beneficial effects of the deep-sea fish detection device provided by the present application are the same as the beneficial effects of the deep-sea fish detection method provided by the above-mentioned embodiment, and the other technical features of the deep-sea fish detection device are the same as the features disclosed in the above-mentioned embodiment method, which will not be repeated here.

[0132] The present application provides a deep-sea fish detection device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the deep-sea fish detection method in the above-mentioned embodiment one.

[0133] Reference below Figure 7, which shows a schematic diagram of the structure of a deep-sea fish detection device suitable for implementing the embodiment of the present application. The deep-sea fish detection device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 7 The deep-sea fish detection device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0134] like Figure 7 As shown, the deep-sea fish detection device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. Various programs and data required for the operation of the deep-sea fish detection device are also stored in the RAM 1004. The processing device 1001, the ROM 1002, and the RAM 1004 are connected to each other via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems may be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 1003 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 1009. The communication device 1009 may allow the deep-sea fish detection device to communicate with other devices wirelessly or by wire to exchange data. Although the deep-sea fish detection device with various systems is shown in the figure, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or have alternatively.

[0135] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0136] The deep-sea fish detection equipment provided by the present application adopts the deep-sea fish detection method in the above-mentioned embodiment, which can solve the technical problem of how to improve the intelligence level of deep-sea fish detection and reduce the mortality rate of fish and the manpower and material costs of farmers. Compared with the prior art, the beneficial effects of the deep-sea fish detection equipment provided by the present application are the same as the beneficial effects of the deep-sea fish detection method provided by the above-mentioned embodiment, and the other technical features of the deep-sea fish detection equipment are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0137] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0138] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0139] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, the computer-readable program instructions being used to execute the deep-sea fish school detection method in the above-mentioned embodiment.

[0140] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0141] The computer-readable storage medium may be included in the deep-sea fish detection device; or may exist independently without being assembled into the deep-sea fish detection device.

[0142] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the deep-sea fish detection device, the deep-sea fish detection device: obtains a preset video stream address of the camera; according to the preset video stream address, uses a preset detection model to perform fish scene detection to obtain a detection result, and the preset detection model includes a depth-separable convolution and a lightweight activation function.

[0143] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0144] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0145] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0146] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned deep-sea fish detection method, and can solve the technical problem of how to improve the intelligence level of deep-sea fish detection and reduce the mortality rate of fish and the manpower and material costs of farmers. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the deep-sea fish detection method provided by the above-mentioned embodiment, and will not be repeated here.

[0147] The present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the deep-sea fish school detection method as described above.

[0148] The computer program product provided by the present application can solve the technical problem of how to improve the intelligence level of deep-sea fish school detection and reduce the mortality rate of fish and the manpower and material costs of farmers. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as the beneficial effects of the deep-sea fish school detection method provided by the above embodiment, and will not be repeated here.

[0149] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A method for detecting deep-sea fish schools, characterized in that: The method comprises: Get the preset video stream address of the camera; According to the preset video stream address, a preset detection model is used to perform fish school image detection to obtain a detection result, wherein the preset detection model includes a depthwise separable convolution and a lightweight activation function.

2. The method according to claim 1, characterized in that The step of performing fish school image detection using a preset detection model according to the preset video stream address to obtain a detection result comprises: Upon receiving a start command from the control port, obtaining a real-time fish school image according to the preset video stream address; Inputting the real-time fish school image into a preset detection model; Based on the preset detection model, the real-time fish school image is processed by deep convolution and the lightweight activation function to obtain a deep convolution output; Based on the preset detection model, the deep convolution output is processed by point-by-point convolution and the lightweight activation function to obtain a detection result.

3. The method according to claim 2, characterized in that The step of processing the real-time fish school image based on the preset detection model through deep convolution and the lightweight activation function to obtain a deep convolution output comprises: Obtaining an input channel for the real-time fish school image; Allocating an independent first convolution kernel to the input channel through the preset detection model; Calculating the dot product of the first convolution kernel and the local area of ​​the real-time fish school image to obtain a two-dimensional feature map; The two-dimensional feature map is processed by the lightweight activation function to obtain a deep convolution output.

4. The method according to claim 2, characterized in that The step of processing the deep convolution output by point-by-point convolution and the lightweight activation function based on the preset detection model to obtain the detection result comprises: Configure a second convolution kernel for the deep convolution output through the preset detection model; Performing point-by-point multiplication and accumulation on all channels of the depth convolution output through a second convolution kernel to obtain a convolution feature map; The convolution feature map is processed by the lightweight activation function to obtain a detection result.

5. The method according to claim 1, characterized in that The step of obtaining the preset video stream address of the camera includes: Receive push data from the control port; The video stream address of the camera is obtained from the push data, and the video stream address is selected by the control port according to the distribution instruction.

6. The method according to claim 1, characterized in that The step of performing fish school image detection using a preset detection model according to the preset video stream address to obtain a detection result comprises: Acquire real-time messaging protocol video stream data and identification data in the detection result; Converting the real-time message transmission protocol video stream data into streaming media data to obtain real-time streaming transmission protocol video stream data; The real-time streaming protocol video stream data and the identification data are connected to a front-end device for display.

7. The method according to any one of claims 1 to 6, characterized in that After the step of performing fish school image detection using a preset detection model according to the preset video stream address and obtaining the detection result, the following step further comprises: Obtaining identification data in the detection result; The identification data is associated with a device identification number and a timestamp and stored.

8. A deep-sea fish school detection device, characterized in that: The device comprises: A receiving module is used to obtain the preset video stream address of the camera; The processing module is used to perform fish school image detection according to the preset video stream address using a preset detection model to obtain a detection result, wherein the preset detection model includes a depth-separable convolution and a lightweight activation function.

9. A deep sea fish school detection device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the deep-sea fish school detection method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the deep-sea fish school detection method according to any one of claims 1 to 7 are implemented.

Images