A method and system for monitoring pyrenoid algae based on binocular images

Abstract

The present invention relates to a method and system for monitoring pyrenoid algae based on binocular images, and specifically to the field of biological monitoring. The method collects binocular image data with pyrenoid algae as monitoring targets in real time, establishes an adaptive background model to achieve dynamic adaptive following of monitoring targets, generates a disparity map and generates a depth map based on focal length and baseline distance, constructs three-dimensional information of the pyrenoid algae, extracts relevant features of the pyrenoid algae, constructs a deep learning model to perform probability distribution of the health status of the pyrenoid algae, constructs support vector regression to predict the growth rate of the pyrenoid algae, improves the judgment accuracy of the health status of the algae, uses a generative adversarial network to perform AI model training to generate the best pyrenoid algae image, uses Euclidean distance to calculate the similarity between the best pyrenoid algae image and binocular image data, and more clearly understands the matching degree between the generated image and the actual image.

Description

Technical Field

[0001] The present invention relates to the field of biological monitoring, and more specifically, to a method and system for monitoring pyrenoid algae based on binocular images. Background Art

[0002] With the intensification of human activities, especially the acceleration of industrial production, agricultural fertilization and urbanization, the structure and dynamic changes of biological communities in water bodies are increasingly affected. The rapid development of computer vision and image processing technology has greatly improved the image-based data processing capabilities, and can provide rich three-dimensional information through binocular imaging technology to help more accurately identify and analyze algae in water bodies.

[0003] Traditional algae monitoring technology has poor adaptability under different lighting conditions, resulting in unstable monitoring results. Due to the complexity of the underwater environment, there is a lack of dynamic tracking and monitoring of the monitoring target, and it is unable to respond to algae changes in the underwater environment in a timely manner. It is slow to respond to abnormal algae growth and sudden changes in light, making it difficult to provide real-time warnings. The lack of response models combined for analysis leads to inconsistent monitoring results. Summary of the invention

[0004] In view of the technical problems existing in the prior art, the present invention provides a method and system for monitoring pyrenoid algae based on binocular images to solve the problems raised in the above-mentioned background technology.

[0005] The technical solution of the present invention to solve the above technical problems is as follows: A method for monitoring pyrenoid algae based on binocular images, comprising the following steps:

[0006] S101. Real-time acquisition of binocular image data with pyrenoid algae as the monitoring target, and real-time acquisition of underwater environment illumination information data, and establishment of an adaptive background model to achieve dynamic adaptive following of the monitoring target;

[0007] S102. Use median filtering to remove salt and pepper noise from binocular image data, increase the global contrast of binocular image data by expanding the contrast range, locally balance binocular image data and enhance binocular image data details, and perform white balance correction by adaptive scale adjustment through binocular image data block;

[0008] S103. Use the SAD algorithm to find the absolute difference between the corresponding blocks under different monitoring perspective differences, and perform regional optimal matching with the binocular image data blocks to generate a disparity map, generate a depth map based on the focal length and baseline distance of the left and right devices of the modern dual camera monitoring, and construct the three-dimensional information of the pyrenoid algae;

[0009] S104. Extract relevant features of pyrenoid algae, construct a deep learning model to perform probability distribution of health status of pyrenoid algae, and construct support vector regression to predict the growth rate of pyrenoid algae;

[0010] S105. Aggregate historical data of pyrenoid algae, use generative adversarial network to train AI model to generate the best pyrenoid algae image, use Euclidean distance combined with algae shape features, algae color features, algae texture features and algae density features to calculate the similarity between the best pyrenoid algae image and binocular image data;

[0011] In a preferred embodiment, in S101, a modern dual-camera monitoring left and right-side devices are installed to the underwater environment to collect binocular image data with attached protein nucleus algae as the monitoring target in real time, and a light sensor is connected to collect the light information data of the underwater environment in real time, and the timestamp corresponding to each binocular image data and light information data is marked, and the monitoring angle of the modern dual-camera monitoring left and right-side devices is maintained at 45° with the monitoring target, and the spatial position of the modern dual-camera monitoring left and right-side devices in the underwater environment and the overlap of the monitoring angle are calibrated.

[0012] Furthermore, an adaptive background model is established. The learning rate is dynamically adjusted as the timestamp changes and the size of the illumination information data changes. The difference between the current frame of the binocular image data and the initialization background frame number of the model is compared to realize scene change detection, and the update signal is triggered to accept and update the adaptive background model. The specific formula is:

[0013] B(x,y,t)=[1-β(t)]B(x,y,t-1)+β(t)I(x,y,t)

[0014] Among them, B(x,y,t) represents the pixel value of the binocular image data (x,y) position of the background frame number in the adaptive background model at timestamp t, β(t) represents the learning rate at timestamp t, B(x,y,t-1) represents the pixel value of the binocular image data (x,y) position of the background frame number in the adaptive background model at timestamp t-1, I(x,y,t) represents the pixel value of the binocular image data (x,y) position of the current frame number in the adaptive background model at timestamp t, the foreground frame number is defined by updating the difference between the current frame of the binocular image data and the current background frame number of the model, the foreground and background of the binocular image data are distinguished by the change in the illumination information data, and dynamic adaptive following and monitoring targets are achieved.

[0015] In a preferred embodiment, in S102, the salt and pepper noise of the binocular image data is removed by using a median filter, the global contrast of the binocular image data is increased by expanding the contrast range, the binocular image data is locally balanced and the details of the binocular image data are enhanced, the visual effect is improved by adjusting the Gamma value of the binocular image data, the white balance correction of the binocular image data is performed by adaptive scale adjustment through the binocular image data, and the comprehensive scaling factor of the color channel is calculated based on the local average and global average of the binocular image data. The specific formula is:

[0016]

[0017] Among them, Scale i represents the comprehensive scaling factor, N represents the number of pixels in the binocular image data block, and w R 、w G 、w B Represent the adaptive weights of the red, green, and blue color channels, respectively. Represents the pixel values ​​of the binocular image data blocks at the (x, y) position of the red, green, and blue color channels. The binocular image data blocks are adaptively adjusted locally in different color channels through the scaling factor and merged into the binocular image data after white balance correction.

[0018] In a preferred embodiment, in S103, for each pixel value of the binocular image data collected by the modern dual camera monitoring left device, the SAD algorithm is used to find the absolute difference between the current pixel value and the pixel value of its neighborhood and the corresponding block of the binocular image data collected by the modern dual camera monitoring right device under different monitoring viewing angle differences, and the specific formula is:

[0019]

[0020] Among them, C SAD (d,x,y) represents the absolute difference, w represents the block rectangle size of the binocular image data, I L (u,v) represents the pixel grayscale value of the binocular image data collected by the left device of the modern dual-camera monitoring at the position (u,v), α L It represents the pixel gray value weight of the binocular image data collected by the left device of the modern dual camera monitoring, I R (ud,v) represents the pixel grayscale value at the disparity d corresponding to the position (u,v) of the binocular image data collected by the modern dual-camera monitoring device on the right side, α RIt represents the pixel gray value weight of the binocular image data collected by the modern dual camera monitoring device on the right side, and performs regional optimal matching with the binocular image data blocks. The disparity map is generated based on the regional optimal matching result, and the depth map is generated based on the focal length and baseline distance of the left and right devices monitored by the modern dual camera. The specific formula is:

[0021]

[0022] Among them, Z(x,y) represents the position where the depth map is generated, f represents the focal length of the left and right devices monitored by the modern dual camera, B represents d(x,y) represents the baseline distance of the left and right devices monitored by the modern dual camera, and c represents the adjustable parameters of the nonlinear related illumination information data. The foreground and background in the monitoring field of view are extracted through regional segmentation and the three-dimensional information of the pyrenoid algae is constructed.

[0023] In a preferred embodiment, in S104, the depth map is converted into a binary image and morphological operations are performed, the algae shape features of the pyrenoid algae are extracted through OpenCV contour detection, the algae color features and algae texture features are extracted through the block-by-block hue and saturation distribution characteristics of the binocular image data, and the algae density features are extracted by combining the depth map and different color channels.

[0024] Furthermore, a deep learning model is constructed, binocular image data and three-dimensional information are received through the input layer, multiple convolution layers are added, ReLU is used as the activation function layer to introduce nonlinearity, and the algae shape features, algae color features, algae texture features and algae density features are summarized and extracted through the convolution kernel, and the algae shape features, algae color features, algae texture features and algae density features are downsampled through the pooling layer, and the first convolution and pooling operations are repeated. The algae shape features, algae color features, algae texture features and algae density features are combined with the marked timestamps to form a time series through the LSTM layer, and the time series is received through the input gate. The time series is gradually passed to the LSTM layer. In each time step, the LSTM layer receives the input algae shape features, algae color features, algae texture features, algae density features and the hidden state of the previous time step, and generates the output algae shape features, algae color features, algae texture features, algae density features and the hidden state of the current time step. The output time series features of the LSTM are received by the fully connected layer and mapped to the probability distribution of the health status of the pyrenoid algae. The softmax function is used to normalize the probability distribution of the health status of the pyrenoid algae and obtain the predicted probability of each health state.

[0025] Furthermore, a support vector regression is constructed, and a matrix containing binocular image data and algae shape features, algae color features, algae texture features, and algae density features is input. The matrix is ​​mapped to a high-dimensional space relying on the Gaussian radial basis kernel, and an error tolerance mechanism is introduced to allow support vector regression to predict deviations with illumination information data errors. Constraints are added to ensure that the deviation between the predicted growth rate of pyrenoid algae and the actual growth rate of pyrenoid algae is within the tolerance range. Illumination information data is added to predict the growth rate of pyrenoid algae. The specific formula is:

[0026] P=w T φ(p,q)+b

[0027] Among them, P represents the predicted value of the growth rate of pyrenoid algae, φ(p,q) represents the nonlinear mapping function that converts the input matrix p and the light information data q into a high-dimensional space, and b represents the bias term of support vector regression.

[0028] In a preferred embodiment, in S105, historical data of pyrenoid algae are aggregated and a database is established, a generative adversarial network is used to perform AI model training to generate the best pyrenoid algae image, and the similarity between the best pyrenoid algae image and the binocular image data is calculated by combining the algae shape features, algae color features, algae texture features, and algae density features of the binocular image data using the Euclidean distance, and the specific formula is:

[0029]

[0030] Where d(j,k) represents the Euclidean distance between the best pyrenoid algae image and the binocular image data, j i represents the best feature of pyrenoid algae image j in the i-th dimension, k i represents the algae shape features, algae color features, algae texture features and algae density features of the binocular image data k in the i-th dimension, n represents the total number of comparison dimensions, f(t) represents the timestamp, and δ represents the nonlinear related illumination information data.

[0031] In a preferred embodiment, it includes an acquisition configuration module, a data preprocessing module, a three-dimensional matching module, a prediction classification module and an adaptive comparison module;

[0032] Acquisition configuration module: Install modern dual-camera monitoring left and right devices to the underwater environment to collect binocular image data with protein nucleus algae as the monitoring target in real time, connect the light sensor to collect the light information data of the underwater environment in real time, establish an adaptive background model to achieve dynamic adaptive following of the monitoring target, and adjust the exposure time and gain settings of the modern dual-camera monitoring left and right devices;

[0033] Data preprocessing module: Use median filtering to remove salt and pepper noise from binocular image data, increase the global contrast of binocular image data by expanding the contrast range, locally balance binocular image data and enhance binocular image data details, and perform white balance correction with adaptive scale adjustment by dividing binocular image data into blocks;

[0034] Three-dimensional matching module: for each pixel value of the binocular image data collected by the left device of the modern dual-camera monitoring, the SAD algorithm is used to find the absolute difference between the current pixel value and the pixel value of its neighborhood and the corresponding block of the binocular image data collected by the right device of the modern dual-camera monitoring under different monitoring viewing angle differences, and the disparity map is generated by performing regional optimal matching with the binocular image data blocks, and the depth map is generated based on the focal length and baseline distance of the left and right devices of the modern dual-camera monitoring. The foreground and background in the monitoring viewing angle are extracted through regional segmentation, and the three-dimensional information of the pyrenoid algae is constructed;

[0035] Prediction and classification module: convert the depth map into a binary image and perform morphological operations to extract the relevant features of the pyrenoid algae, build a deep learning model for the probability distribution of the health status of the pyrenoid algae, and build a support vector regression to predict the growth rate of the pyrenoid algae;

[0036] Adaptive comparison module: Aggregate historical data of pyrenoid algae and establish a database, use generative adversarial network to train AI model to generate the best pyrenoid algae image, use Euclidean distance combined with algae shape features, algae color features, algae texture features and algae density features to calculate the similarity between the best pyrenoid algae image and binocular image data;

[0037] The beneficial effects of the present invention are as follows: the dynamic behavior of the pyrenoid algae can be monitored in real time through a binocular camera, so that the system can timely capture the dynamically changing underwater environment; the monitoring parameters can be automatically adjusted according to different underwater lighting conditions through the collection of real-time lighting information, thereby improving the image quality and the accuracy of target detection; the targets appearing in the changing environment can be identified and tracked by establishing an adaptive background model, ensuring accurate distinction between background and foreground; the exposure time and gain setting of the camera can be automatically adjusted to ensure that the best image effect can be obtained under different lighting conditions, thereby improving the clarity and detail presentation of the image; the salt and pepper noise can be effectively removed by using a median filter to maintain The edges and details of the image can improve the clarity and readability of the image, and the overall brightness difference of the image can be enhanced by expanding the contrast range. The visual effect of the image can be improved by adjusting the Gamma value, making the dark details clearer and the bright parts less likely to be lost, thereby improving the overall visual appeal. The accuracy of judging the health status of algae can be improved by building a deep learning model combined with binocular image data and three-dimensional information. The growth rate of algae can be efficiently predicted by introducing support vector regression. The best image of algae with attached protein nucleus can be obtained through GAN training. The Euclidean distance provides a multi-dimensional and multi-feature similarity measurement method to more clearly understand the degree of match between the generated image and the actual image. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 is a flow chart of the method of the present invention;

[0039] Figure 2 This is a system structure block diagram of the present invention. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0041] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, "plurality" means two or more, unless otherwise clearly and specifically defined.

[0042] In the description of the present application, the term "for example" is used to mean "used as an example, illustration or description". Any embodiment described as "for example" in the present application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is given to enable any technician in the field to implement and use the present invention. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present invention can be implemented without using these specific details. In other examples, well-known structures and processes will not be elaborated in detail to avoid unnecessary details to obscure the description of the present invention. Therefore, the present invention is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in the present application.

[0043] Example 1

[0044] This embodiment provides Figure 1 A method for monitoring pyrenoid algae based on binocular images is shown, which specifically includes the following steps:

[0045] S101. Real-time acquisition of binocular image data with pyrenoid algae as the monitoring target, and real-time acquisition of underwater environment illumination information data, and establishment of an adaptive background model to achieve dynamic adaptive following of the monitoring target;

[0046] S102. Use median filtering to remove salt and pepper noise from binocular image data, increase the global contrast of binocular image data by expanding the contrast range, locally balance binocular image data and enhance binocular image data details, and perform white balance correction by adaptive scale adjustment through binocular image data block;

[0047] S103. Use the SAD algorithm to find the absolute difference between the corresponding blocks under different monitoring perspective differences, and perform regional optimal matching with the binocular image data blocks to generate a disparity map, generate a depth map based on the focal length and baseline distance of the left and right devices of the modern dual camera monitoring, and construct the three-dimensional information of the pyrenoid algae;

[0048] S104. Extract relevant features of pyrenoid algae, construct a deep learning model to perform probability distribution of health status of pyrenoid algae, and construct support vector regression to predict the growth rate of pyrenoid algae;

[0049] S105. Aggregate historical data of pyrenoid algae, use generative adversarial network to train AI model to generate the best pyrenoid algae image, use Euclidean distance combined with algae shape features, algae color features, algae texture features and algae density features to calculate the similarity between the best pyrenoid algae image and binocular image data;

[0050] Example 2

[0051] This embodiment provides Figure 2 The present invention discloses a monitoring system for pyrenoid algae based on binocular images, which specifically includes: an acquisition configuration module, a data preprocessing module, a three-dimensional matching module, a prediction and classification module, and an adaptive comparison module;

[0052] Acquisition configuration module: Install modern dual-camera monitoring left and right devices to the underwater environment to collect binocular image data with protein nucleus algae as the monitoring target in real time, connect the light sensor to collect the light information data of the underwater environment in real time, establish an adaptive background model to achieve dynamic adaptive following of the monitoring target, and adjust the exposure time and gain settings of the modern dual-camera monitoring left and right devices;

[0053] Data preprocessing module: Use median filtering to remove salt and pepper noise from binocular image data, increase the global contrast of binocular image data by expanding the contrast range, locally balance binocular image data and enhance binocular image data details, and perform white balance correction with adaptive scale adjustment by dividing binocular image data into blocks;

[0054] Three-dimensional matching module: for each pixel value of the binocular image data collected by the left device of the modern dual-camera monitoring, the SAD algorithm is used to find the absolute difference between the current pixel value and the pixel value of its neighborhood and the corresponding block of the binocular image data collected by the right device of the modern dual-camera monitoring under different monitoring viewing angle differences, and the disparity map is generated by performing regional optimal matching with the binocular image data blocks, and the depth map is generated based on the focal length and baseline distance of the left and right devices of the modern dual-camera monitoring. The foreground and background in the monitoring viewing angle are extracted through regional segmentation, and the three-dimensional information of the pyrenoid algae is constructed;

[0055] Prediction and classification module: convert the depth map into a binary image and perform morphological operations to extract the relevant features of the pyrenoid algae, build a deep learning model for the probability distribution of the health status of the pyrenoid algae, and build a support vector regression to predict the growth rate of the pyrenoid algae;

[0056] Adaptive comparison module: Aggregate historical data of pyrenoid algae and establish a database, use generative adversarial network to train AI model to generate the best pyrenoid algae image, use Euclidean distance combined with algae shape features, algae color features, algae texture features and algae density features to calculate the similarity between the best pyrenoid algae image and binocular image data;

[0057] S101. Real-time acquisition of binocular image data with pyrenoid algae as the monitoring target, and real-time acquisition of underwater environment illumination information data, and establishment of an adaptive background model to achieve dynamic adaptive following of the monitoring target;

[0058] Furthermore, modern dual-camera monitoring left and right-side devices are installed in the underwater environment to collect binocular image data with protein-nuclear algae as the monitoring target in real time, and the light sensor is connected to collect the light information data of the underwater environment in real time, and the timestamps corresponding to each binocular image data and light information data are marked to ensure that the observation area with protein-nuclear algae is covered, and the monitoring angle of the modern dual-camera monitoring left and right-side devices is kept at 45° to the monitoring target, and the spatial position of the modern dual-camera monitoring left and right-side devices in the underwater environment and the overlap of the monitoring angle are calibrated to ensure the overlap and alignment of the binocular images at different perspectives of the same scene.

[0059] Furthermore, an adaptive background model is established, the average number of binocular image data initialization background frames is used, and the initial value of the learning rate is set to 0.01. The learning rate is dynamically adjusted as the timestamp changes and the size of the illumination information data changes. The difference between the current frame of the binocular image data and the initialization background frame number of the model is compared to realize scene change detection, and the update signal is triggered to accept and update the adaptive background model. The specific formula is:

[0060] B(x,y,t)=[1-β(t)]B(x,y,t-1)+β(t)I(x,y,t)

[0061] Among them, B(x,y,t) represents the pixel value of the binocular image data (x,y) position of the background frame number in the adaptive background model at timestamp t, β(t) represents the learning rate at timestamp t, B(x,y,t-1) represents the pixel value of the binocular image data (x,y) position of the background frame number in the adaptive background model at timestamp t-1, I(x,y,t) represents the pixel value of the binocular image data (x,y) position of the current frame number in the adaptive background model at timestamp t, the foreground frame number is defined by updating the difference between the current frame of the binocular image data and the current background frame number of the model, the foreground and background of the binocular image data are distinguished by the change in the illumination information data, and dynamic adaptive following and monitoring targets are achieved.

[0062] Furthermore, based on the adaptive background model, the exposure time and gain settings of the modern dual camera monitoring the left and right devices are adjusted. When the illumination information data is set to 10000lx, the current underwater environment is a strong light condition, the exposure time of the modern dual camera monitoring the left and right devices is selected to be 1 / 1000s, and the gain setting of the modern dual camera monitoring the left and right devices is selected to be ISO100. When the illumination information data is set to 500lx, the current underwater environment is a strong light condition, the exposure time of the modern dual camera monitoring the left and right devices is selected to be 1 / 30s, and the gain setting of the modern dual camera monitoring the left and right devices is selected to be ISO800.

[0063] S102. Use median filtering to remove salt and pepper noise from binocular image data, increase the global contrast of binocular image data by expanding the contrast range, locally balance binocular image data and enhance binocular image data details, and perform white balance correction by adaptive scale adjustment through binocular image data block;

[0064] Furthermore, the salt and pepper noise of the binocular image data is removed by using median filtering, the global contrast of the binocular image data is increased by expanding the contrast range, the binocular image data is locally balanced and the details of the binocular image data are enhanced, the visual effect is improved by adjusting the Gamma value of the binocular image data, the white balance correction of the adaptive scale adjustment is performed by blocking the binocular image data, and the comprehensive scaling factor of the color channel is calculated based on the local average and global average of the binocular image data. The specific formula is:

[0065]

[0066] Among them, Scale i represents the comprehensive scaling factor, N represents the number of pixels in the binocular image data block, and w R 、w G 、w B Represent the adaptive weights of the red, green, and blue color channels, respectively. Represents the pixel values ​​of the binocular image data blocks at the (x, y) position of the red, green, and blue color channels. The binocular image data blocks are adaptively adjusted locally in different color channels through the scaling factor and merged into the binocular image data after white balance correction.

[0067] S103. Use the SAD algorithm to find the absolute difference between the corresponding blocks under different monitoring perspective differences, and perform regional optimal matching with the binocular image data blocks to generate a disparity map, generate a depth map based on the focal length and baseline distance of the left and right devices of the modern dual camera monitoring, and construct the three-dimensional information of the pyrenoid algae;

[0068] Furthermore, for each pixel value of the binocular image data collected by the left device of the modern dual-camera monitoring, the SAD algorithm is used to find the absolute difference between the current pixel value and the pixel value of its neighborhood and the corresponding block of the binocular image data collected by the right device of the modern dual-camera monitoring under different monitoring viewing angle differences. The specific formula is:

[0069]

[0070] Among them, C SAD (d,x,y) represents the absolute difference, w represents the block rectangle size of the binocular image data, I L(u,v) represents the pixel grayscale value of the binocular image data collected by the left device of the modern dual-camera monitoring at the position (u,v), α L It represents the pixel gray value weight of the binocular image data collected by the left device of the modern dual camera monitoring, I R (ud,v) represents the pixel grayscale value at the disparity d corresponding to the position (u,v) of the binocular image data collected by the modern dual-camera monitoring device on the right side, α R It represents the pixel gray value weight of the binocular image data collected by the modern dual camera monitoring device on the right side, and performs regional optimal matching with the binocular image data blocks. A disparity map is generated based on the regional optimal matching result, where the pixel value of each disparity map represents the disparity value of the monitoring target in the scene. A depth map is generated based on the focal length and baseline distance of the left and right devices of the modern dual camera monitoring device. The specific formula is:

[0071]

[0072] Among them, Z(x,y) represents the position where the depth map is generated, f represents the focal length of the left and right devices monitored by the modern dual camera, B represents d(x,y) represents the baseline distance of the left and right devices monitored by the modern dual camera, and c represents the adjustable parameters of the nonlinear related illumination information data. The foreground and background in the monitoring field of view are extracted through regional segmentation and the three-dimensional information of the pyrenoid algae is constructed.

[0073] S104. Extract relevant features of pyrenoid algae, construct a deep learning model to perform probability distribution of health status of pyrenoid algae, and construct support vector regression to predict the growth rate of pyrenoid algae;

[0074] Furthermore, the depth map is converted into a binary image and morphological operations are performed. The erosion operation is used to remove small boundary objects of the three-dimensional information of the pyrenoid algae. The dilation operation is used to fill the internal holes of the area of ​​​​the three-dimensional information of the pyrenoid algae. The algae shape features of the pyrenoid algae are extracted through OpenCV contour detection. The algae color features and algae texture features are extracted through the distribution characteristics of the block hue and saturation of the binocular image data. The algae density features are extracted by combining the depth map and different color channels.

[0075] Furthermore, a deep learning model is constructed, binocular image data and three-dimensional information are received through the input layer, multiple convolution layers are added, nonlinearity is introduced using ReLU as the activation function layer, and algae shape features, algae color features, algae texture features, and algae density features are summarized and extracted through the convolution kernel, and the algae shape features, algae color features, algae texture features, and algae density features are downsampled through the pooling layer, and the first convolution and pooling operations are repeated to extract high-level features of the binocular image data. A batch normalization layer is added to each convolution layer, and the binocular image data of each channel in the convolution kernel is normalized. The algae shape features, algae color features, algae texture features, and algae density features are combined with the marked timestamp to form a time matrix through the LSTM layer. Sequence, where each time step represents the algae shape feature, algae color feature, algae texture feature and algae density feature extracted by a recurrent neural network, receives the time series through the input gate, and passes the time series step by step to the LSTM layer. In each time step, the input algae shape feature, algae color feature, algae texture feature and algae density feature of the current time step and the hidden state of the previous time step are received through the LSTM layer, and the output algae shape feature, algae color feature, algae texture feature and algae density feature of the current time step and the hidden state of the current time step are generated. Each LSTM layer in the long short-term memory network generates an output time series feature, where the output of each time step contains the time series information of the time step, and receives the LSTM layer through the fully connected layer. The output time series characteristics of TM are mapped to the probability distribution of the health status of the pyrenoid algae. The softmax function is used to normalize the probability distribution of the health status of the pyrenoid algae and obtain the predicted probability of each health status, denoted as M. The health status of the pyrenoid algae is divided into healthy, stressed and endangered. When M≤35%, the health status of the pyrenoid algae is classified as healthy; when 35%<M≤75%, the health status of the pyrenoid algae is classified as stressed; when 75%<M, the health status of the pyrenoid algae is classified as endangered.

[0076] Furthermore, a support vector regression is constructed, and a matrix containing binocular image data and algae shape features, algae color features, algae texture features, and algae density features is input. The matrix is ​​mapped to a high-dimensional space relying on the Gaussian radial basis kernel, and an error tolerance mechanism is introduced to allow support vector regression to predict deviations with illumination information data errors. Constraints are added to ensure that the deviation between the predicted growth rate of pyrenoid algae and the actual growth rate of pyrenoid algae is within the tolerance range. Illumination information data is added to predict the growth rate of pyrenoid algae. The specific formula is:

[0077] P=w T φ(p,q)+b

[0078] Among them, P represents the predicted value of the growth rate of pyrenoid algae, φ(p,q) represents the nonlinear mapping function that converts the input matrix p and the light information data q into a high-dimensional space, and b represents the bias term of support vector regression.

[0079] S105. Aggregate historical data of pyrenoid algae, use generative adversarial network to train AI model to generate the best pyrenoid algae image, use Euclidean distance combined with algae shape features, algae color features, algae texture features and algae density features to calculate the similarity between the best pyrenoid algae image and binocular image data;

[0080] Furthermore, the historical data of pyrenoid algae are aggregated and a database is established. The generative adversarial network is used to train the AI ​​model to generate the best pyrenoid algae image. The Euclidean distance is combined with the algae shape features, algae color features, algae texture features and algae density features of the binocular image data to calculate the similarity between the best pyrenoid algae image and the binocular image data. The specific formula is:

[0081]

[0082] Where d(j,k) represents the Euclidean distance between the best pyrenoid algae image and the binocular image data, j i represents the best feature of pyrenoid algae image j in the i-th dimension, k i represents the algae shape features, algae color features, algae texture features and algae density features of the binocular image data k in the i-th dimension, n represents the total comparison dimension, f(t) represents the timestamp, δ represents the nonlinear related illumination information data, wherein the smaller the Euclidean distance is, the greater the similarity between the best protein nucleus algae image and the binocular image data is, that is, the difference between the best protein nucleus algae image and the binocular image data is small, and the larger the Euclidean distance is, the smaller the similarity between the best protein nucleus algae image and the binocular image data is, that is, the difference between the best protein nucleus algae image and the binocular image data is large.

[0083] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0084] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0085] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0086] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0087] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0088] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0089] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for monitoring pyrenoid algae based on binocular images, characterized in that: The specific steps include: S101. Real-time acquisition of binocular image data with pyrenoid algae as the monitoring target, and real-time acquisition of underwater environment illumination information data, and establishment of an adaptive background model to achieve dynamic adaptive following of the monitoring target; The learning rate is dynamically adjusted as the timestamp changes and the size of the illumination information data changes. The difference between the current frame of the binocular image data and the initialization background frame of the model is compared to detect scene changes, trigger an update signal, and accept and update the adaptive background model. S102. Use median filtering to remove salt and pepper noise from binocular image data, increase the global contrast of binocular image data by expanding the contrast range, locally balance binocular image data and enhance binocular image data details, and perform white balance correction by adaptive scale adjustment through binocular image data block; S103. Use the SAD algorithm to find the absolute difference between the corresponding blocks under different monitoring perspective differences, and perform regional optimal matching with the binocular image data blocks to generate a disparity map, generate a depth map based on the focal length and baseline distance of the left and right devices of the modern dual camera monitoring, and construct the three-dimensional information of the pyrenoid algae; S104. Extract relevant features of pyrenoid algae, construct a deep learning model to perform probability distribution of health status of pyrenoid algae, and construct support vector regression to predict the growth rate of pyrenoid algae; S105. Aggregate historical data of pyrenoid algae, use generative adversarial networks to train AI models to generate the best pyrenoid algae images, and use Euclidean distance combined with algae shape features, algae color features, algae texture features, and algae density features to calculate the similarity between the best pyrenoid algae images and binocular image data.

2. The method for monitoring pyrenoid algae based on binocular images according to claim 1, characterized in that: In the S101, a modern dual camera is installed to monitor the left and right devices to the underwater environment to collect binocular image data with the attached protein nucleus algae as the monitoring target in real time, and a light sensor is connected to collect the light information data of the underwater environment in real time, and the timestamp corresponding to each binocular image data and light information data is marked to establish an adaptive background model.

3. The method for monitoring pyrenoid algae based on binocular images according to claim 2, characterized in that: The adaptive background model dynamically adjusts the learning rate as the timestamp changes in combination with the change in the illumination information data, compares the difference between the current frame of the binocular image data and the initialization background frame number of the model to realize scene change detection, and triggers an update signal to accept and update the adaptive background model. The specific formula is: B(x,y,t)=[1-β(t)]B(x,y,t-1)+β(t)I(x,y,t) Among them, B(x,y,t) represents the pixel value of the binocular image data (x,y) position of the background frame number in the adaptive background model at timestamp t, β(t) represents the learning rate at timestamp t, B(x,y,t-1) represents the pixel value of the binocular image data (x,y) position of the background frame number in the adaptive background model at timestamp t-1, I(x,y,t) represents the pixel value of the binocular image data (x,y) position of the current frame number in the adaptive background model at timestamp t, the foreground frame number is defined by updating the difference between the current frame of the binocular image data and the current background frame number of the model, the foreground and background of the binocular image data are distinguished by the change in the illumination information data, and dynamic adaptive following and monitoring targets are achieved.

4. The method for monitoring pyrenoid algae based on binocular images according to claim 1, characterized in that: In S102, the salt and pepper noise of the binocular image data is removed by using a median filter, the global contrast of the binocular image data is increased by expanding the contrast range, the binocular image data is locally balanced and the details of the binocular image data are enhanced, the visual effect is improved by adjusting the Gamma value of the binocular image data, white balance correction of adaptive scale adjustment is performed on the binocular image data in blocks, a comprehensive scaling factor of the color channel is calculated based on the local average value and the global average value of the binocular image data in blocks, and adaptive local adjustment of different color channels is performed on the binocular image data in blocks by the scaling factor and the blocks are merged into binocular image data after white balance correction.

5. The method for monitoring pyrenoid algae based on binocular images according to claim 4, characterized in that: The specific formula for calculating the comprehensive scaling factor of the color channel based on the block local average and global average of the binocular image data is: Among them, Scale i represents the comprehensive scaling factor, N represents the number of pixels in the binocular image data block, and w R 、w G 、w B Represent the adaptive weights of the red, green, and blue color channels, respectively. Represents the pixel value of the block of binocular image data at the red, green, and blue color channel (x, y) position.

6. The method for monitoring pyrenoid algae based on binocular images according to claim 1, characterized in that: In S103, for each pixel value of the binocular image data collected by the modern dual camera monitoring left device, the SAD algorithm is used to find the absolute difference between the current pixel value and the pixel value of its neighborhood and the corresponding block of the binocular image data collected by the modern dual camera monitoring right device under different monitoring viewing angle differences, and the specific formula is: Among them, C SAD (d,x,y) represents the absolute difference, w represents the block rectangle size of the binocular image data, I L (u,v) represents the pixel grayscale value of the binocular image data collected by the left device of the modern dual-camera monitoring at the position (u,v), α L It represents the pixel gray value weight of the binocular image data collected by the left device of the modern dual camera monitoring, I R (ud,v) represents the pixel grayscale value at the disparity d corresponding to the position (u,v) of the binocular image data collected by the modern dual-camera monitoring device on the right side, α R It represents the pixel gray value weight of the binocular image data collected by the modern dual camera monitoring the right device, and performs regional optimal matching with the binocular image data blocks. The disparity map is generated according to the regional optimal matching result, and the depth map is generated based on the focal length and baseline distance of the left and right devices monitored by the modern dual camera. The specific formula is: Among them, Z(x,y) represents the position where the depth map is generated, f represents the focal length of the left and right devices monitored by the modern dual camera, B represents the baseline distance of the left and right devices monitored by the modern dual camera, d(x,y) represents the relative position difference under the viewing angle of the modern dual camera, and c represents the adjustable parameters of the nonlinear related illumination information data. The foreground and background in the monitoring viewing angle are extracted through regional segmentation, and the three-dimensional information of the pyrenoid algae is constructed.

7. The method for monitoring pyrenoid algae based on binocular images according to claim 1, characterized in that: In the S104, a deep learning model is constructed, binocular image data and three-dimensional information are received through an input layer, a plurality of convolution layers are added, nonlinearity is introduced by using ReLU as an activation function layer, and algae shape features, algae color features, algae texture features, and algae density features are summarized and extracted through convolution kernels, algae shape features, algae color features, algae texture features, and algae density features are downsampled through a pooling layer, the first convolution and pooling operations are repeated, the algae shape features, algae color features, algae texture features, and algae density features are combined with a marked timestamp to form a time series through an LSTM layer, and the time series is received through an input gate, The time series is passed step by step to the LSTM layer. In each time step, the input algae shape features, algae color features, algae texture features, algae density features and the hidden state of the previous time step are received through the LSTM layer, and the output algae shape features, algae color features, algae texture features, algae density features and the hidden state of the current time step are generated. The output time series features of the LSTM are received through the fully connected layer and mapped to the probability distribution of the health state of the pyrenoid algae. The probability distribution of the health state of the pyrenoid algae is normalized using the softmax function to obtain the predicted probability of each health state.

8. The method for monitoring pyrenoid algae based on binocular images according to claim 1, characterized in that: In the S104, a support vector regression is constructed, a matrix including binocular image data and algae shape features, algae color features, algae texture features, and algae density features is input, and the matrix is ​​mapped to a high-dimensional space relying on a Gaussian radial basis kernel, an error tolerance mechanism is introduced to allow support vector regression to predict deviations with illumination information data errors, constraints are added to ensure that the deviation between the predicted growth rate of pyrenoid algae and the actual growth rate of pyrenoid algae is within a tolerance range, and illumination information data is added to predict the growth rate of pyrenoid algae, and the specific formula is: P=w T φ(p,q)+b Where P represents the predicted value of the growth rate of pyrenoid algae, w T The corresponding transpose of the matrix representing the algae shape features, algae color features, algae texture features, and algae density features. φ(p,q) represents the nonlinear mapping function that converts the input matrix p and the illumination information data q into a high-dimensional space. b represents the bias term of support vector regression.

9. The method for monitoring pyrenoid algae based on binocular images according to claim 1, characterized in that: In S105, historical data of pyrenoid algae are aggregated and a database is established. A generative adversarial network is used to train an AI model to generate the best pyrenoid algae image. The similarity between the best pyrenoid algae image and the binocular image data is calculated by combining the algae shape features, algae color features, algae texture features, and algae density features of the binocular image data using the Euclidean distance. The specific formula is: Where d(j,k) represents the Euclidean distance between the best pyrenoid algae image and the binocular image data, j i represents the best feature of pyrenoid algae image j in the i-th dimension, k i represents the algae shape features, algae color features, algae texture features and algae density features of the binocular image data k in the i-th dimension, n represents the total number of comparison dimensions, f(t) represents the timestamp, and δ represents the nonlinear related illumination information data.

10. A binocular image-based pyrenoid algae monitoring system is applied to a binocular image-based pyrenoid algae monitoring method as claimed in any one of claims 1 to 9, characterized in that: It includes acquisition configuration module, data preprocessing module, three-dimensional matching module, prediction classification module and adaptive comparison module; Acquisition configuration module: Install modern dual-camera monitoring left and right devices to the underwater environment to collect binocular image data with protein nucleus algae as the monitoring target in real time, connect the light sensor to collect the light information data of the underwater environment in real time, establish an adaptive background model to achieve dynamic adaptive following of the monitoring target, and adjust the exposure time and gain settings of the modern dual-camera monitoring left and right devices; The learning rate is dynamically adjusted as the timestamp changes and the size of the illumination information data changes. The difference between the current frame of the binocular image data and the initialization background frame of the model is compared to detect scene changes, trigger an update signal, and accept and update the adaptive background model. Data preprocessing module: Use median filtering to remove salt and pepper noise from binocular image data, increase the global contrast of binocular image data by expanding the contrast range, locally balance binocular image data and enhance binocular image data details, and perform white balance correction with adaptive scale adjustment by dividing binocular image data into blocks; Three-dimensional matching module: for each pixel value of the binocular image data collected by the left device of the modern dual-camera monitoring, the SAD algorithm is used to find the absolute difference between the current pixel value and the pixel value of its neighborhood and the corresponding block of the binocular image data collected by the right device of the modern dual-camera monitoring under different monitoring viewing angle differences, and the disparity map is generated by performing regional optimal matching with the binocular image data blocks, and the depth map is generated based on the focal length and baseline distance of the left and right devices of the modern dual-camera monitoring. The foreground and background in the monitoring viewing angle are extracted through regional segmentation, and the three-dimensional information of the pyrenoid algae is constructed; Prediction and classification module: convert the depth map into a binary image and perform morphological operations to extract the relevant features of the pyrenoid algae, build a deep learning model for the probability distribution of the health status of the pyrenoid algae, and build a support vector regression to predict the growth rate of the pyrenoid algae; Adaptive comparison module: Aggregate historical data of pyrenoid algae and establish a database, use generative adversarial networks to train AI models to generate the best pyrenoid algae images, and use Euclidean distance combined with algae shape features, algae color features, algae texture features, and algae density features to calculate the similarity between the best pyrenoid algae images and binocular image data.

Images