Crayfish quality detection method and device oriented to complex environment

Abstract

The invention provides a crayfish quality detection method and device oriented to a complex environment, and belongs to the field of aquatic product detection.The method comprises the steps that target image data of crayfishes are collected, environment characteristics of the same time and the same place are obtained, and the environment characteristics comprise the temperature, the illumination intensity and the noise intensity; obtaining a crayfish skeleton model according to the target image data, and extracting morphological size features including length, width and area; fusing the environment features and the size features to obtain a multi-dimensional feature vector, inputting the multi-dimensional feature vector into a trained quality detection model, and outputting a quality category; wherein the quality detection model is obtained by training according to target image data of different environment features and size features of pre-marked quality categories. The method can effectively reduce interference caused by environmental noise and improve the accuracy of field crayfish quality detection.

Description

Technical Field

[0001] The present invention relates to the field of aquatic product detection, and in particular to a crayfish quality detection method and device for complex environments. Background Art

[0002] As an important aquatic product, the quality assessment of crayfish is crucial to improving market value and consumer satisfaction. However, the traditional manual grading method is inefficient and highly subjective.

[0003] Currently, there are methods for quality detection based on crayfish images and convolutional neural networks. However, these methods are not only inaccurate, but also unable to make timely and accurate quality judgments on crayfish after capture in the wild. For crayfish caught at the breeding site, quality detection is particularly important, not only can immature crayfish be put back for continued breeding, but also individuals that do not meet quality standards can be screened out, thereby reducing transportation and maintenance costs. Summary of the invention

[0004] In view of the problems existing in the prior art, the present invention provides a crayfish quality detection method and device for complex environments.

[0005] The present invention provides a crayfish quality detection method for complex environments, comprising: collecting crayfish target image data, and obtaining environmental characteristics at the same time and place, wherein the environmental characteristics include temperature, light intensity and noise intensity; obtaining a crayfish skeleton model according to the target image data, and extracting morphological size characteristics, wherein the size characteristics include length, width and area; obtaining a multi-dimensional feature vector according to the fusion of the environmental characteristics and the size characteristics, inputting the multi-dimensional feature vector into a trained quality detection model, and outputting a quality category; wherein the quality detection model is obtained after training based on target image data with different environmental characteristics and size characteristics that are pre-labeled with quality categories.

[0006] According to a crayfish quality detection method for complex environments provided by the present invention, the quality categories include size categories and integrity categories. Accordingly, after outputting the quality categories, it also includes: determining the comprehensive quality category of the detected crayfish based on the size category and the integrity category, and outputting the comprehensive quality category.

[0007] According to a crayfish quality detection method for complex environments provided by the present invention, before collecting the crayfish target image data, it also includes: collecting real-time image data of crayfish; inputting the real-time image data into a trained image quality assessment model, and outputting the image quality score of the real-time image data; repeating the process of adjusting camera parameters according to the image quality score, collecting real-time image data after adjusting the parameters, inputting the trained image quality assessment model, and outputting the image quality score of the real-time image data, until the output image quality score reaches a preset threshold; using the real-time image data that reaches the preset threshold as the target image data; wherein the camera parameters include exposure, and the image quality assessment model is obtained after training based on crayfish images labeled with image quality scores.

[0008] According to a crayfish quality detection method for complex environments provided by the present invention, the crayfish skeleton model is obtained according to the target image data, and the morphological size features are extracted, including: inputting the target image into a trained key point detection model, and outputting the key points of the crayfish image; according to the key point positions, based on a morphological skeletonization algorithm, extracting the crayfish skeleton model, the skeleton model including the head and tail main axis and the claw secondary axis; based on the skeleton model and all key point positions, polygon fitting is performed to obtain the outer contour, and the size features are determined according to the outer contour.

[0009] According to a crayfish quality detection method for complex environments provided by the present invention, the quality category includes a completeness category. Accordingly, before collecting crayfish target image data, it also includes: inputting the target image data to be labeled with quality categories into the trained key point detection model, and outputting the key points of the crayfish image; extracting the skeleton model of the crayfish based on the morphological skeletonization algorithm according to the key point positions; performing polygon fitting based on the skeleton model and all key point positions to obtain an outer contour, and determining a size feature based on the outer contour; determining a completeness type label based on the ratio of the area to the square of the length in the size feature and labeling it, and training the quality detection model based on the target image data labeled with the completeness type label.

[0010] According to a crayfish quality detection method for complex environments provided by the present invention, the multi-dimensional feature vector is input into a trained quality detection model, comprising: forming a feature graph from the multi-dimensional feature vectors of multiple crayfish, and inputting the feature graph into a quality detection model based on a convolutional neural network.

[0011] According to a crayfish quality detection method for complex environments provided by the present invention, the multi-dimensional feature vector is input into a trained quality detection model, comprising: forming a feature graph with the multi-dimensional feature vectors of multiple crayfish, inputting the feature graph into a quality detection model based on a convolutional neural network, and outputting quality categories of size and completeness respectively; wherein the quality detection model is constructed based on a multi-task learning framework.

[0012] According to a crayfish quality detection method for complex environments provided by the present invention, before obtaining the crayfish skeleton model according to the target image data and extracting the morphological size features, it also includes: calculating the color temperature and color cast parameters of the ambient light source based on the light intensity and based on a color constancy model; and performing white balance correction on the image based on the gray world hypothesis and an edge-preserving filter according to the color temperature and the color cast parameters.

[0013] The present invention also provides a crayfish quality detection device for complex environments, comprising: an acquisition module, used to collect crayfish target image data, and obtain environmental characteristics at the same time and place, the environmental characteristics including temperature, light intensity and noise intensity; an extraction module, used to obtain a crayfish skeleton model according to the target image data, and extract morphological size characteristics, the size characteristics including length, width and area; a processing module, used to obtain a multi-dimensional feature vector based on the fusion of the environmental characteristics and the size characteristics, input the multi-dimensional feature vector into a trained quality detection model, and output a quality category; wherein the quality detection model is obtained after training based on target image data with different environmental characteristics and size characteristics that are pre-labeled with quality categories.

[0014] The present invention also provides a crayfish quality detection system for complex environments, comprising: a conveyor belt, a conveyor belt control unit, a posture adjustment mechanism, an industrial camera, a temperature sensor, a light sensor, a noise analyzer and the above-mentioned device; the conveyor belt is used to transport shrimp bodies to be inspected, and the conveyor belt control unit is used to control the movement of the conveyor belt; the posture adjustment mechanism is used to adjust the spatial position and angle of the shrimp bodies to be inspected; the industrial camera is fixed on a bracket, and is used to take images of the shrimp bodies when the shrimp bodies to be inspected reach a fixed position through the conveyor belt, and send the images to the device; the temperature sensor, light sensor, and noise analyzer are used to collect ambient temperature, light intensity, and ambient noise, and send them to the device.

[0015] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for detecting quality of crayfish in a complex environment as described above is implemented.

[0016] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the method for detecting the quality of crayfish in a complex environment as described in any one of the above is implemented.

[0017] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-mentioned crayfish quality detection methods for complex environments.

[0018] The crayfish quality detection method and device for complex environments provided by the present invention perform quality detection through environmental characteristics such as temperature, light intensity and noise intensity combined with size characteristics, which can effectively reduce the interference caused by environmental noise and improve the accuracy of quality detection of crayfish in the wild. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 This is one of the flow diagrams of the crayfish quality detection method for complex environments provided by the present invention;

[0021] Figure 2 This is the second flow chart of the crayfish quality detection method for complex environments provided by the present invention;

[0022] Figure 3 It is a schematic diagram of the structure of the crayfish quality detection device for complex environments provided by the present invention;

[0023] Figure 4 It is a structural schematic diagram of a crayfish quality detection system for complex environments provided by the present invention;

[0024] Figure 5 is a schematic diagram of the structure of an electronic device provided by the present invention;

[0025] Figure numerals: 1. conveyor belt; 2. noise analyzer; 3. posture adjustment mechanism; 4. temperature sensor; 5. industrial camera; 6. conveyor belt control unit. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] The present invention proposes an automatic detection method and grading device for crayfish quality based on multi-sensor fusion technology, which integrates high-precision sensors such as ambient temperature, light intensity, and ambient noise, and combines advanced image processing technology to achieve automatic detection and grading of crayfish quality. The device can detect and grade crayfish quality immediately after being caught in the wild, greatly improving grading efficiency and accuracy.

[0028] Combine the following Figure 1-Figure 5 The invention describes a crayfish quality detection method and device for complex environments. Figure 1 This is one of the flow diagrams of the crayfish quality detection method for complex environments provided by the present invention, such as Figure 1 As shown, the present invention provides a crayfish quality detection method for complex environments, comprising:

[0029] 101. Collect crayfish target image data and obtain environmental characteristics at the same time and place, wherein the environmental characteristics include temperature, light intensity, and noise intensity.

[0030] First, you can choose a multifunctional industrial camera, such as Hikvision's MV-CA050-20GC model, which has a resolution of 5 million pixels (2448×2048), providing detailed image resolution, and can clearly capture the subtle features of crayfish (such as shell patterns, gill status, etc.). The frame rate can reach up to 20 frames per second, the pixel size is 3.45μm×3.45μm, and the sensitivity is 5000mV@1 / 30s. The built-in zoom lens can achieve an optical zoom function of 1× to 4×. By performing real-time analysis of the dynamic area and posture of crayfish, the zoom ratio is automatically adjusted to ensure that high-resolution images with rich details can be obtained for individuals of different sizes. At the same time, optical zoom is combined with electronic image stabilization technology to effectively avoid image jitter and blur caused by changes in wind speed in the wild.

[0031] The camera module is combined with an efficient image encoder (such as H.265) to transmit high-resolution images to the master control system of the grading device in real time with low bandwidth requirements. Data is transmitted via optical fiber to ensure lossless image data and low latency during transmission.

[0032] In addition, a set of high-precision ambient temperature sensors, such as DS18B20, can be deployed in combination with new thermocouples to monitor ambient temperature in real time and collect data through a single-chip microcomputer. Adaptive filtering algorithms can be used to dynamically adjust temperature data to eliminate instantaneous fluctuations, while machine learning algorithms can be used to identify outliers to ensure data reliability.

[0033] Among them, the temperature measurement formula using the DS18B20 sensor is:

[0034]

[0035] Where D is the raw data returned by the sensor.

[0036] For thermocouples, the temperature can be calculated using the cold junction compensation formula:

[0037] T actual =T measured -T reference

[0038] Among them, T reference is the cold end temperature.

[0039] Ambient light intensity data can be obtained in real time through light sensors (such as BH1750). In addition, a noise analyzer with a built-in sensor can be introduced, and an adaptive noise suppression algorithm can be used for real-time noise monitoring. A threshold is set, and when the noise exceeds the set threshold, the system automatically collects noise data. All noise data is processed by FFT and combined with other sensor data to form a full range of environmental features. The built-in sensor captures the sound in the surrounding environment to indirectly measure the jitter of the device during the field detection and grading process. For example, when the wind is strong, the jitter of the device will affect the image quality.

[0040] 102. Acquire a crayfish skeleton model according to the target image data, and extract morphological size features, where the size features include length, width, and area.

[0041] In order to accurately measure the size of crayfish, the size characteristics must be determined first. The size characteristics can be determined through feature points. First, it is necessary to identify its important feature points (such as head, tail, claws, etc.), and then extract the skeletal structure of the crayfish. The main feature points of crayfish can be identified using a key point detection model based on deep learning (such as HRNet or OpenPose). By marking the key points of different parts of the crayfish in the training data set, the model can accurately identify the position of the head, abdomen, tail and claws, and further determine the size characteristics.

[0042] 103. A multi-dimensional feature vector is obtained by fusing the environmental feature and the size feature, and the multi-dimensional feature vector is input into a trained quality detection model to output a quality category.

[0043] The crayfish size features (such as length, width, area, etc.) obtained by image analysis are fused with environmental perception data (such as temperature, light, noise) to construct a multi-dimensional feature vector F.

[0044] F=[L,W,V,T,L int ,N] (5)

[0045] Where L is the length of the crayfish; W is the width of the crayfish; V is the area of ​​the crayfish; T is the ambient temperature; L int is the light intensity; N is the ambient noise level.

[0046] The quality category can be a category or a specific value, such as a parameter of size category or size value.

[0047] In one embodiment, the quality category includes size category and completeness. Accordingly, after outputting the quality category, it also includes: determining the comprehensive quality category of the detected crayfish according to the size category and the completeness category, and outputting the comprehensive quality category.

[0048] The completeness is the degree of body loss, or the ratio of the actual body size to the body size that should be achieved in morphology. For example, the completeness of a shrimp with full claws is 100% or high, and the completeness of a shrimp with one claw missing or one claw deformed is 80% or medium. Specifically, the feature range of different quality grades can be determined by clustering the statistical information in the training data (such as K-means clustering). Then, the quality category labels are annotated and the quality detection model is trained. For example, the output layer is set according to the quality indicators of crayfish, and the size is divided into three categories, "large", "medium" and "small", and the completeness is also divided into three categories, "high", "medium" and "low".

[0049] For different scenarios, the comprehensive quality category can be determined according to the requirements of the size category and the completeness category, and then the crayfish can be machine-screened based on this. For example, for those with high completeness requirements, the completeness category of "high" and the size categories of "large" and "medium" can be used as qualified in the comprehensive quality category to screen the crayfish. For scenarios with high size requirements, only the size category of "high" can be used as qualified in the comprehensive quality category to screen the crayfish. Of course, the comprehensive quality category is not limited to qualified and unqualified, and multiple levels can also be customized.

[0050] The crayfish quality detection method for complex environments of the present invention performs quality detection through environmental characteristics such as temperature, light intensity and noise intensity combined with size characteristics, which can effectively reduce the interference caused by environmental noise and improve the accuracy of quality detection of crayfish in the wild.

[0051] In one embodiment, before collecting the target image data of crayfish, it also includes: collecting real-time image data of crayfish; inputting the real-time image data into a trained image quality assessment model, and outputting an image quality score of the real-time image data; repeating the process of adjusting camera parameters according to the image quality score, collecting real-time image data after adjusting the parameters, inputting the trained image quality assessment model, and outputting the image quality score of the real-time image data, until the output image quality score reaches a preset threshold; using the real-time image data that reaches the preset threshold as the target image data; wherein the camera parameters include exposure, and the image quality assessment model is obtained after training based on crayfish images with annotated image quality scores.

[0052] In order to automatically evaluate the quality of the image, the present invention adopts an image quality evaluation method based on deep learning to obtain the above-mentioned target image data. Specifically, a pre-trained convolutional neural network (such as VGG or ResNet) can be used to score the quality of the image. The network model can detect problems such as blur, overexposure or underexposure in the image and generate a quality score. According to the score, it is judged whether the current exposure setting or other camera parameters are reasonable, and the camera parameters are dynamically adjusted as needed to ensure that the quality of the final image meets the expected standards. For example, if the quality score is <0.4: the image is severely blurred; the score is 0.4-0.6: the image is slightly blurred; the score is >0.6: the image is clear; the score is <0.35: the image is severely overexposed; the score is 0.35-0.55: slightly overexposed; the score is >0.55: normal exposure. The camera parameters are adjusted based on the image quality score, and the obtained camera parameters are used to obtain a new real-time image, and this is repeated until the quality score meets the preset threshold. The real-time image obtained at this time is used as the target image in step 101, and the environmental characteristics at that time are collected for the quality detection of the subsequent steps 102 and 103.

[0053] The crayfish quality detection method for complex environments of the embodiment of the present invention performs quality scoring on real-time images according to an image quality assessment model, and adjusts camera parameters to obtain new real-time images. Through dynamic cyclic adjustment, images that meet expected standards can be captured, thereby improving the accuracy of size feature extraction and further improving the accuracy of quality detection.

[0054] In one embodiment, the method of acquiring a crayfish skeleton model and extracting morphological size features based on the target image data includes: inputting the target image into a trained key point detection model to output the key points of the crayfish image; extracting the crayfish skeleton model based on a morphological skeletonization algorithm according to the key point positions, wherein the skeleton model includes a main axis of the head and tail and a secondary axis of the claws; performing polygon fitting based on the skeleton model and all key point positions to obtain an outer contour, and determining the size features based on the outer contour.

[0055] Specifically, the skeleton of the crayfish is extracted using a morphological skeletonization algorithm. Based on the positions of key points, a skeleton model containing the main axis of the head and tail and the secondary axis of the claws is generated. Combining the geometric relationship between the main axis and the secondary axis, the basic dimensions of the crayfish, such as the length and width, can be inferred. After extracting the skeleton structure of the crayfish, the size of the crayfish is accurately analyzed by further combining morphological and geometric calculation methods.

[0056] The formula for calculating the length of the main axis is:

[0057]

[0058] Among them, (x1, y1) and (x2, y2) are the two endpoints of the main axis.

[0059] Select two symmetrical points on the minor axis, and the width can be expressed as:

[0060] W = d(p1, p2)

[0061] Among them, p1 and p2 are two key points on the secondary axis, and d represents the distance between them.

[0062] The minimum enclosing rectangle algorithm is used to calculate the enclosing rectangle of the crayfish. The length and width of the rectangle can be used to preliminarily estimate the length and width of the crayfish. To improve accuracy, the enclosing rectangle can be rotated and aligned according to the direction of the main axis of the skeleton to ensure that the long side of the enclosing rectangle is parallel to the main axis of the crayfish.

[0063] In the embodiment of the present invention, the outline of the crayfish is fitted with a polygon to extract its precise outline. By calculating the vertex distance of the polygon, the accurate measurement of the length and width of the crayfish is further improved. The problem of low accuracy of the conventional convolutional network model is avoided. For crayfish with complex shapes (such as the claws are unfolded), the angle and degree of unfolding of the claws can be calculated, and the actual length can be compensated according to the results.

[0064] In one embodiment, the quality category includes a completeness category. Accordingly, before collecting the crayfish target image data, it also includes: inputting the target image data to be labeled with the quality category into the trained key point detection model, and outputting the key points of the crayfish image; extracting the skeleton model of the crayfish based on the morphological skeletonization algorithm according to the key point positions; performing polygon fitting based on the skeleton model and all key point positions to obtain the shape contour, and determining the size feature based on the shape contour; determining the completeness type label according to the ratio of the area to the square of the length in the size feature and labeling it, and training the quality detection model based on the target image data labeled with the completeness type label.

[0065] The training process of the quality detection model requires target images with labeled quality categories as training samples. However, considering the labeling of conventional completeness categories, it is usually based on manual labeling. Manual labeling either determines the category subjectively or determines the category by measuring the length. These methods are labor-intensive, highly subjective and inaccurate.

[0066] In the embodiment of the present invention, the training samples of the quality detection model are processed by the above-mentioned morphological skeletonization algorithm to obtain the area and length, and then the integrity type label is determined, which is not only efficient but also accurate. The method of marking the integrity in the embodiment of the present invention is not limited to the integrity level caused by conventional limb loss when identifying the quality category, but can also classify morphologically incomplete shrimp bodies, thereby improving the final effect of quality detection.

[0067] In one embodiment, inputting the multi-dimensional feature vector into the trained quality detection model includes: forming a feature graph from the multi-dimensional feature vectors of a plurality of crayfish, and inputting the feature graph into a quality detection model based on a convolutional neural network.

[0068] In an embodiment of the present invention, for the fused multi-dimensional feature vector, deep learning is used to establish a quality detection model (such as a convolutional neural network, CNN) to evaluate the overall quality of crayfish (such as size, completeness, etc.). A convolutional neural network (such as ResNet, VGG, etc.) is used for image feature extraction, and the model consists of multiple convolutional layers, pooling layers, and fully connected layers. For the input data, a feature map is constructed based on the multi-dimensional feature vectors of multiple crayfish. For example, the multi-dimensional feature vectors of 20 crayfish constitute a 20×6 feature map. In addition, the multi-dimensional feature vector is not limited to the above 6 features, and other key features can be added, such as constituting a 20×10 feature map. For each quality category, different convolutional network models can be used for training and prediction. For example, for size categories and completeness, two convolutional network models are used to construct a size quality detection model and a completeness quality detection model.

[0069] In addition, for the various features (size, completeness, etc.) in the quality assessment of crayfish, a multi-layer feature fusion strategy can be introduced. Through cross-layer connection and feature pyramid structure, the features extracted from different convolutional layers are fused to improve the perception of multi-scale and multi-dimensional features. Another advantage of feature fusion is that it can combine local and global information, so that the model can capture the overall characteristics of crayfish while not ignoring subtle local differences.

[0070] The crayfish quality detection method of the embodiment of the present invention uses a convolutional neural network to predict the quality of a multi-dimensional feature vector, which can avoid the problem of poor fitting effect of linear regression models, support vector machine models, etc., while taking into account the influence of the correlation between different crayfish, making the results of variety detection more accurate.

[0071] In one embodiment, inputting the multi-dimensional feature vector into the trained quality detection model includes: forming a feature map from the multi-dimensional feature vectors of multiple crayfish, inputting the feature map into a quality detection model based on a convolutional neural network, and outputting quality categories of size and completeness respectively; wherein, the quality detection model is constructed based on a multi-task learning framework.

[0072] In an embodiment of the present invention, in order to simultaneously evaluate multiple quality dimensions of crayfish (such as size, morphology, etc.), a multi-task learning framework can be used. Under this framework, different tasks such as size regression and damage detection (loss detection can be based on integrity value, or integrity value combined with size) share the underlying convolutional features, and at the same time, predictions are made through different output branches at the top layer. Each task branch uses a different loss function: for example, the size regression task uses the mean squared error (MSE), and the damage detection task uses Focal Loss to deal with sample imbalance problems.

[0073] In one embodiment, the method of acquiring a crayfish skeleton model according to the target image data and extracting morphological size features also includes: calculating the color temperature and color cast parameters of the ambient light source according to the light intensity based on a color constancy model; and performing white balance correction on the image based on the grayscale world hypothesis and an edge-preserving filter according to the color temperature and the color cast parameters.

[0074] Considering that images may have color cast or color distortion problems in complex lighting environments, we first calculate the color temperature and color cast parameters of the ambient light source based on the collected light intensity data, and then perform white balance correction on the image to eliminate color deviation and restore the true color, thereby improving the accuracy of quality detection.

[0075] In addition, other preprocessing operations can be performed. Figure 2 As shown:

[0076] Multi-scale image enhancement: To further improve the image quality under different lighting conditions, the image is decomposed into detail layers and base layers of different scales using multi-scale decomposition based on the Laplacian pyramid according to the collected light intensity data. The histogram equalization algorithm is applied to the base layer to enhance the overall contrast; while for the detail layer, nonlinear enhancement methods such as Laplacian filtering and adaptive sharpening are used to enhance the local details of the image.

[0077] Dynamic background modeling and target segmentation: In order to better segment the target area of ​​crayfish under different lighting conditions, a background model can be established through a mixed Gaussian model to update background changes in real time. When segmenting the target, a convolutional neural network is combined to accurately classify the foreground and background, especially for scenes with complex backgrounds and large lighting changes. This method can effectively improve the accuracy of target detection and segmentation.

[0078] Noise suppression and detail preservation: In low-light conditions, images are prone to noise, which affects the accuracy of crayfish quality assessment. To this end, a joint bilateral filtering and non-local mean filtering algorithm can be used to remove noise while retaining the edge and detail information of the image to the greatest extent. In addition, in the denoising process, deep learning models such as convolutional neural networks are combined to automatically identify and remove common noise patterns through learning and training of a large amount of crayfish image data, further improving the clarity of the image.

[0079] The crayfish quality detection device for complex environments provided by the present invention is described below. The crayfish quality detection device for complex environments described below and the crayfish quality detection method for complex environments described above can be referenced to each other.

[0080] Figure 3 Schematic diagram of the structure of the crayfish quality detection device for complex environments provided by the present invention. Figure 3 As shown, the crayfish quality detection device for complex environments includes: an acquisition module 301, an extraction module 302 and a processing module 303. Among them, the acquisition module 301 is used to collect crayfish target image data and obtain environmental characteristics at the same time and place, and the environmental characteristics include temperature, light intensity and noise intensity; the extraction module 302 is used to obtain the crayfish skeleton model according to the target image data, and extract morphological size characteristics, and the size characteristics include length, width and area; the processing module 303 is used to obtain a multi-dimensional feature vector according to the fusion of the environmental characteristics and the size characteristics, input the multi-dimensional feature vector into the trained quality detection model, and output the quality category; wherein, the quality detection model is obtained after training according to the target image data of different environmental characteristics and size characteristics with quality categories pre-labeled.

[0081] The device embodiments provided in the embodiments of the present invention are for implementing the above-mentioned method embodiments. For specific processes and detailed contents, please refer to the above-mentioned method embodiments, which will not be repeated here.

[0082] The crayfish quality detection device for complex environments provided in the embodiment of the present invention has the same implementation principle and technical effects as those of the aforementioned embodiment of the crayfish quality detection method for complex environments. For the sake of brief description, for matters not mentioned in the embodiment of the crayfish quality detection device for complex environments, reference may be made to the corresponding contents in the aforementioned embodiment of the crayfish quality detection method for complex environments.

[0083] Figure 4 Schematic diagram of the structure of the crayfish quality detection system for complex environments provided by the present invention. Figure 4 The present invention also provides a crayfish quality detection system for complex environments, comprising: a conveyor belt 1, a conveyor belt control unit 6, a posture adjustment mechanism 3, an industrial camera 5, a temperature sensor 4, a light sensor, a noise analyzer 2 and the above-mentioned crayfish quality detection device for complex environments; the conveyor belt 1 is used to transport shrimp bodies to be inspected, and the conveyor belt control unit 6 is used to control the movement of the conveyor belt; the posture adjustment mechanism 3 is used to adjust the spatial position and angle of the shrimp bodies to be inspected; the industrial camera 5 is fixed on a bracket, and is used to take images of the shrimp bodies when the shrimp bodies to be inspected reach a fixed position through the conveyor belt 1, and send the images to the device; the temperature sensor 4, the light sensor, and the noise analyzer 2 are used to collect ambient temperature, light intensity and ambient noise, and send them to the device.

[0084] The industrial camera can be Hikvision's multifunctional industrial camera, MV-CA050-20GC, with a resolution of 5 million pixels (2448×2048), providing detailed image resolution, and can clearly capture the subtle features of crayfish (such as shell patterns, gill status, etc.). The frame rate can reach up to 20 frames per second, the pixel size is 3.45μm×3.45μm, and the sensitivity is 5000mV@1 / 30s.

[0085] The conveyor belt control unit 6 can adopt a PLC control unit, which is a universal component and its function is to control the movement of the conveyor belt. The function of the conveyor belt 1 is mainly to transport the shrimp bodies to be inspected so that the shrimp bodies can reach each designated position for inspection. The specific processing process and technical effects can be referred to the above method embodiment, which will not be repeated here.

[0086] Figure 5 is a schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 5As shown, the electronic device may include: a processor 501, a communication interface 502, a memory 503 and a communication bus 504, wherein the processor 501, the communication interface 502 and the memory 503 communicate with each other through the communication bus 504. The processor 501 may call the logic instructions in the memory 503 to execute the crayfish quality detection method for complex environments, the method comprising: collecting crayfish target image data, and obtaining environmental features at the same time and place, wherein the environmental features include temperature, light intensity and noise intensity; obtaining a crayfish skeleton model according to the target image data, extracting morphological size features, wherein the size features include length, width and area; obtaining a multi-dimensional feature vector according to the fusion of the environmental features and the size features, inputting the multi-dimensional feature vector into the trained quality detection model, and outputting a quality category; wherein the quality detection model is obtained after training according to the target image data of different environmental features and size features with quality categories pre-labeled.

[0087] In addition, the logic instructions in the above-mentioned memory 503 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0088] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the crayfish quality detection method for complex environments provided by the above methods, the method including: collecting crayfish target image data, and obtaining environmental characteristics at the same time and place, the environmental characteristics including temperature, light intensity and noise intensity; obtaining a crayfish skeleton model according to the target image data, and extracting morphological size features, the size features including length, width and area; obtaining a multi-dimensional feature vector by fusing the environmental characteristics and the size features, inputting the multi-dimensional feature vector into a trained quality detection model, and outputting a quality category; wherein the quality detection model is obtained after training based on target image data with different environmental characteristics and size characteristics that are pre-labeled with quality categories.

[0089] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the crayfish quality detection method for complex environments provided by the above-mentioned methods, the method comprising: collecting crayfish target image data, and obtaining environmental characteristics at the same time and place, the environmental characteristics including temperature, light intensity and noise intensity; obtaining a crayfish skeleton model according to the target image data, and extracting morphological size characteristics, the size characteristics including length, width and area; obtaining a multi-dimensional feature vector by fusing the environmental characteristics and the size characteristics, inputting the multi-dimensional feature vector into a trained quality detection model, and outputting a quality category; wherein the quality detection model is obtained after training based on target image data with different environmental characteristics and size characteristics that are pre-labeled with quality categories.

[0090] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0091] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0092] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A crayfish quality detection method for complex environments, characterized in that: include: Collecting crayfish target image data and obtaining environmental characteristics at the same time and place, wherein the environmental characteristics include temperature, light intensity and noise intensity; Acquire a crayfish skeleton model according to the target image data, and extract morphological size features, wherein the size features include length, width and area; A multi-dimensional feature vector is obtained by fusing the environmental feature and the size feature, the multi-dimensional feature vector is input into a trained quality detection model, and a quality category is output; The quality detection model is obtained through training based on target image data with different environmental features and size features that are pre-labeled with quality categories.

2. The crayfish quality detection method for complex environments according to claim 1, characterized in that: The quality categories include size categories and integrity categories. Accordingly, after the output quality categories, the following are also included: According to the size category and the completeness category, the comprehensive quality category of the detected crayfish is determined, and the comprehensive quality category is output.

3. The crayfish quality detection method for complex environments according to claim 1, characterized in that: Before collecting the crayfish target image data, the method further includes: Collect real-time image data of crayfish; Inputting the real-time image data into a trained image quality assessment model, and outputting an image quality score of the real-time image data; Repeat the process of adjusting the camera parameters according to the image quality score, collecting real-time image data after adjusting the parameters, inputting the trained image quality assessment model, and outputting the image quality score of the real-time image data until the output image quality score reaches a preset threshold; Using the real-time image data reaching a preset threshold as the target image data; The camera parameters include exposure, and the image quality assessment model is obtained after training with crayfish images labeled with image quality scores.

4. The crayfish quality detection method for complex environments according to claim 1, characterized in that: The step of obtaining a crayfish skeleton model according to the target image data and extracting morphological size features includes: Input the target image into the trained key point detection model, and output the key points of the crayfish image; According to the key point positions, based on the morphological skeletonization algorithm, a skeleton model of the crayfish is extracted, wherein the skeleton model includes a head-tail main axis and a claw secondary axis; Based on the skeleton model and all key point positions, polygon fitting is performed to obtain an outer contour, and size features are determined according to the outer contour.

5. The crayfish quality detection method for complex environments according to claim 4 is characterized in that: The quality category includes a completeness category. Accordingly, before collecting the crayfish target image data, the method further includes: Input the trained key point detection model into the target image data to be labeled with quality categories, and output the key points of the crayfish image; According to the key point positions, the skeleton model of crayfish is extracted based on the morphological skeletonization algorithm; Based on the skeleton model and all key point positions, polygon fitting is performed to obtain an outline, and size features are determined according to the outline; According to the ratio of the area to the square of the length in the size feature, the integrity type label is determined and annotated, and the quality detection model is trained based on the target image data annotated with the integrity type label.

6. The crayfish quality detection method for complex environments according to claim 1, characterized in that: The step of inputting the multi-dimensional feature vector into the trained quality detection model comprises: The multi-dimensional feature vectors of a plurality of crayfish are used to form a feature map, and the feature map is input into a quality detection model based on a convolutional neural network.

7. The crayfish quality detection method for complex environments according to claim 2, characterized in that: The step of inputting the multi-dimensional feature vector into the trained quality detection model comprises: The multi-dimensional feature vectors of the plurality of crayfish are used to form a feature map, and the feature map is input into a quality detection model based on a convolutional neural network to output the quality categories of size and completeness respectively; Among them, the quality detection model is built based on a multi-task learning framework.

8. The crayfish quality detection method for complex environments according to claim 1, characterized in that: Before obtaining the crayfish skeleton model according to the target image data and extracting the morphological size features, the method further includes: According to the light intensity, based on a color constancy model, the color temperature and color cast parameters of the ambient light source are calculated; According to the color temperature and the color cast parameter, based on a gray world hypothesis and an edge preserving filter, white balance correction is performed on the image.

9. A crayfish quality detection device for complex environments, characterized in that: include: An acquisition module is used to acquire the crayfish target image data and obtain the environmental characteristics at the same time and place, wherein the environmental characteristics include temperature, light intensity and noise intensity; An extraction module, used to obtain a crayfish skeleton model according to the target image data, and extract morphological size features, wherein the size features include length, width and area; A processing module, configured to obtain a multi-dimensional feature vector by fusing the environmental feature and the size feature, input the multi-dimensional feature vector into a trained quality detection model, and output a quality category; The quality detection model is obtained through training based on target image data with different environmental features and size features that are pre-labeled with quality categories.

10. A crayfish quality detection system for complex environments, characterized in that: include: A conveyor belt, a conveyor belt control unit, a posture adjustment mechanism, an industrial camera, a temperature sensor, a light sensor, a noise analyzer, and the device according to claim 9; The conveyor belt is used to transport the shrimp bodies to be inspected, and the conveyor belt control unit is used to control the movement of the conveyor belt; The posture adjustment mechanism is used to adjust the spatial position and angle of the shrimp body to be inspected; The industrial camera is fixed on the bracket and is used to take images of the shrimp bodies when the shrimp bodies to be inspected pass through the conveyor belt and arrive at a fixed position, and send the images to the device; The temperature sensor, light sensor and noise analyzer are used to collect ambient temperature, light intensity and ambient noise, and send them to the device.

Images