Hatching egg gender detection system based on multi-modal information fusion

Abstract

The invention provides a hatching egg gender detection system based on multi-modal information fusion, and belongs to the field of agricultural product detection, and the system comprises a detection box which is of a closed cavity structure with an openable door; the supporting mechanism is fixedly installed at the top of the weighing table, is used for adjusting the posture of the hatching eggs placed on the supporting mechanism and comprises a hatching egg supporting frame, a buffering piece and a rotating piece; the weighing platform is arranged in the detection box and is used for acquiring weight information of the hatching eggs; the multi-angle camera is used for obtaining the volume information of the hatching eggs according to the three-dimensional reconstruction; the infrared laser thermometer is positioned on the side surface of the hatching egg and is used for acquiring an infrared image of the hatching egg; the transmission camera is used for acquiring a transmission image of the hatching egg to be detected under the irradiation of the transmission light source and is mounted right above the hatching egg; and the detection module is used for determining the gender type of the hatching egg according to the weight information, the volume information, the transmission image and the infrared image. According to the system, hatching egg gender characteristics are excavated from multiple dimensions, the gender detection accuracy and reliability are greatly improved, and stable supporting is achieved through the supporting mechanism to avoid interference.

Description

Technical Field

[0001] This invention belongs to the field of agricultural product testing, specifically relating to a sex detection system for hatching eggs based on multimodal information fusion. Background Technology

[0002] In large-scale poultry farming, sex selection of hatching eggs is a crucial preliminary step. For layer chickens, male eggs fail to lay after hatching, resulting in a significant waste of hatching, feed, and space resources. For broiler chickens, different sexes exhibit differences in growth rate and feed conversion rate; pre-screening allows for separate rearing and improves farming efficiency. Therefore, pre-hatching sex testing of hatching eggs is of great significance for reducing farming costs and improving the industry's economic benefits.

[0003] Several methods for detecting the sex of hatching eggs have emerged in related technical fields, such as acoustic detection, optical detection, and near-infrared spectroscopy. However, individual differences in hatching eggs (such as shell thickness and air cell size) can interfere with the detection signal. Detection methods based on a single information dimension have weak anti-interference capabilities, limited accuracy, and are difficult to adapt to complex actual breeding scenarios. In addition, the detection environment of existing detection equipment lacks effective control. External interference factors such as light, temperature, and vibration can easily affect the acquisition accuracy of sensor signals, further reducing detection accuracy. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a sex detection system for hatching eggs based on multimodal information fusion.

[0005] This invention provides a hatching egg sex detection system based on multimodal information fusion, including a detection box, a weighing platform, an infrared laser thermometer, a transmission light source, a multi-angle camera, a transmission camera, a detection module, and a hatching egg sex detection support mechanism as described above. The testing box is a closed cavity structure with an openable and closable door; the egg sex detection support mechanism is fixedly installed on the top of the weighing platform to adjust the posture of the eggs placed on it. The support mechanism includes a pair of egg support frames, each egg support frame including a pair of brackets, a pair of buffers installed on the corresponding brackets, and a rotating component disposed between the pair of buffers; the buffers include a damper and a spring, the damper including a housing and a telescopic part, one end of the housing being fixedly connected to the bracket, the telescopic part being slidably connected to the housing and extending beyond the other end of the housing, one end of the spring being connected to the housing and the other end being connected to the telescopic part; the rotating component includes multiple series-connected electric rollers, each electric roller including a motor roller, a shaft and a rubber sleeve, the end of the shaft being hinged to the telescopic part or the shaft of an adjacent electric roller, the motor roller being rotatably installed outside the shaft and driving the rubber sleeve to rotate; The weighing platform is installed inside the testing box to obtain the weight information of the hatching eggs; the multi-angle camera is used to obtain the volume information of the hatching eggs based on three-dimensional reconstruction; the infrared laser thermometer is located on the side of the hatching eggs to obtain infrared images of the hatching eggs; the transmission camera is used to obtain transmission images of the hatching eggs under transmission light source illumination; the detection module is used to determine the sex type of the hatching eggs based on the weight information, the volume information, the transmission image, and the infrared image.

[0006] According to the multimodal information fusion-based hatching egg sex detection system provided by the present invention, the transmission camera is installed directly above the hatching egg position; the multi-angle camera includes a first camera and a second camera; the first camera is installed diagonally above the hatching egg position; and the second camera is installed to the side of the hatching egg position.

[0007] According to the multimodal information fusion-based hatching egg sex detection system provided by the present invention, the transmission image and the infrared image are images of the bleeding stage, and the weighing platform is used to acquire first weight information of the hatching egg at the start of incubation and second weight information of the hatching egg at the bleeding stage; the detection module determines the density loss of the hatching egg at the bleeding stage based on the volume information and in combination with the first weight information and the second weight information; correspondingly, the detection module determines the sex type of the hatching egg based on the density loss, the transmission image and the infrared image.

[0008] According to the multimodal information fusion-based hatching egg sex detection system provided by the present invention, the detection module determines the sex type of the hatching egg based on a sex detection model. The sex detection model includes an image segmentation network, a numerical extraction network, a feature fusion network, and a classification prediction network, wherein: the image segmentation network is used to input a transmission image at the bloodline stage and output a first image feature; the numerical extraction network is used to input density loss and two-dimensional infrared image data and output a second image feature; the feature fusion network is used to fuse the first image feature and the second image feature and input them into the classification prediction network to obtain the sex detection result of the poultry egg; wherein, the sex detection model is trained based on sample hatching eggs with determined sex results.

[0009] According to the multimodal information fusion-based egg sex detection system provided by the present invention, the bloodline period includes the fourth day of egg incubation.

[0010] According to the multimodal information fusion-based egg sex detection system provided by the present invention, the image segmentation network includes a U-Net or ResNet-50 module, and the numerical extraction network includes a fully connected neural network.

[0011] According to the multimodal information fusion-based egg sex detection system provided by the present invention, the sex detection model incorporates a self-attention mechanism.

[0012] According to the multimodal information fusion-based hatching egg sex detection system provided by the present invention, the two buffers of each hatching egg support are tilted upward and inward.

[0013] According to the multimodal information fusion-based egg sex detection system provided by the present invention, the two ends of the rubber sleeve are provided with annular buffer cavities.

[0014] According to the multimodal information fusion-based egg sex detection system provided by the present invention, the side of the rubber sleeve is a concave rotating surface.

[0015] This invention provides a multimodal information fusion-based hatching egg sex detection system. Through a sensing unit composed of a transmission camera, a multi-angle camera, and an infrared laser thermometer, a multimodal information acquisition system is constructed, providing comprehensive data support for accurate sex determination of hatching eggs. The multi-angle camera can capture image information such as the appearance and internal structure of the hatching egg, providing a basis for sex determination based on image features. The infrared laser thermometer can collect temperature data of the hatching egg, using the difference in temperature distribution between male and female eggs to assist in sex determination. Finally, through the fusion acquisition of multimodal information, the limitations of traditional single-information detection methods are overcome, enabling the mining of sex characteristics of hatching eggs from multiple dimensions, significantly improving the accuracy and reliability of sex detection. The support mechanism, through a symmetrical hatching egg support frame, combined with a "damper + spring" buffer structure, and multiple series-connected electric rollers for shape adjustment, can achieve non-destructive and stable support for the hatching egg, and can also achieve precise and automated adjustment of the hatching egg angle through the electric rollers, avoiding vibration interference factors during angle adjustment in existing methods, thus making the detection results more accurate. Attached Figure Description

[0016] The present invention will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings: Figure 1 A three-dimensional structural diagram of the egg sex detection system based on multimodal information fusion provided by the present invention; Figure 2 This is a schematic diagram of the main structure of the egg sex detection system based on multimodal information fusion provided by the present invention; Figure 3 This is a schematic diagram of the structure of the egg sex detection support mechanism provided by the present invention; Figure 4 A schematic diagram of the buffer component and electric idler roller structure provided by the present invention; Figure 5 for Figure 4 Enlarged structural diagram at point A; Figure 6 This is one of the schematic diagrams of the clamping structure of the infrared laser thermometer provided by the present invention; Figure 7 This is the second schematic diagram of the clamping structure of the infrared laser thermometer provided by the present invention; Figure 8 Image of bloodline in hatching eggs provided for this invention.

[0017] Reference numerals: 1-Detection box; 2-Weighing platform; 3-Support mechanism; 31-Bracket; 32-Buffer; 321-Damper; 322-Spring; 33-Electric roller; 331-Motor roller; 332-Shaft; 333-Rubber sleeve; 334-Buffer cavity; 4-Transmitted light source; 5-First electric telescopic rod; 6-Transmitted camera; 7-Second electric telescopic rod; 8-First camera; 9-Third electric telescopic rod; 10-Clamping component; 101-Iron plate; 102-Rubber pad; 103-Anti-slip groove; 104-Rubber strip; 105-Magnet block; 106-Slot; 11-Infrared laser thermometer; 12-Fourth electric telescopic rod; 13-Second camera; 14-Interactive device. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0019] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal communication between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances. Furthermore, in the description of this invention, unless otherwise stated, "a plurality of" or "several" means two or more.

[0020] The following is combined Figures 1-8 This invention describes a hatching egg sex detection system based on multimodal information fusion. For example... Figure 1 , Figure 2 As shown, the egg sex detection system based on multimodal information fusion of the present invention includes: a detection box 1, a weighing platform 2, a support mechanism 3, a transmission light source 4, a transmission camera 6, an infrared laser thermometer 11, a multi-angle camera, and a detection module.

[0021] The detection box 1 is a closed cavity structure with an openable and closable door; the egg sex detection support mechanism 3 is fixedly installed on the top of the weighing platform 2 to adjust the posture of the eggs placed on it; the weighing platform 2 is set inside the detection box 1 to obtain the weight information of the eggs; the multi-angle camera is used to obtain the volume information of the eggs based on three-dimensional reconstruction; the infrared laser thermometer 11 is located on the side of the eggs to obtain infrared images of the eggs; the transmission camera 6 is used to obtain the transmission image of the eggs under transmission light source illumination; the detection module is used to determine the sex type of the eggs based on the weight information, the volume information, the transmission image, and the infrared image.

[0022] The detection chamber 1 provides a stable environment for the entire detection process, effectively isolating external interference factors such as light, temperature, and vibration, and avoiding the impact of the external environment on the accuracy of the detection data, thus laying an environmental foundation for the reliability of subsequent multimodal information acquisition.

[0023] Weighing platform 2 can accurately acquire the weight information data, providing basic physical parameters to support the determination of the sex of hatching eggs. The weight difference can be used as an auxiliary reference dimension for sex differentiation.

[0024] Given that the existing testing equipment has an unreasonable egg support structure design, most of which adopt a fixed posture support method, it is impossible to adjust the angle of the eggs according to the testing needs. As a result, some embryonic characteristics cannot be effectively captured. At the same time, the support structure lacks a buffer protection mechanism, which can easily cause mechanical damage to the eggs during placement and adjustment, affecting the integrity of the eggs and the subsequent hatching effect.

[0025] like Figure 3 , 4 As shown in Figure 5, the support mechanism 3 of the present invention includes a pair of egg support frames, each egg support frame including a pair of brackets 31, a pair of buffer members 32 mounted on the corresponding brackets, and a rotating member disposed between the pair of buffer members 32.

[0026] The buffer 32 includes a damper 321 and a spring 322. The damper 321 includes a housing and a telescopic part. One end of the housing is fixedly connected to the bracket 31, and the telescopic part is slidably connected to the housing and extends out of the other end of the housing. One end of the spring 322 is connected to the housing, and the other end is connected to the telescopic part.

[0027] The rotating components include multiple electrically driven rollers 33 connected in series. Each electrically driven roller 33 includes a motor roller 331, a shaft 332, and a rubber sleeve 333. The end of the shaft 332 is hinged to the shaft 332 of the telescopic part or an adjacent electrically driven roller. The motor roller 331 is rotatably mounted outside the shaft 332 and drives the rubber sleeve 333 to rotate. A pair of rotating components together form an ellipse that can support the hatching eggs.

[0028] Specifically, four brackets 31 can be welded to the four corners of the weighing part of the weighing platform 2 to form a symmetrical support structure, which can provide a stable installation base for the hatching egg support frame, avoid the hatching eggs from shifting due to the shaking of the brackets 31 during the testing process, and ensure the consistency of the testing position.

[0029] The buffer element 32 can be welded to the upper end of the bracket 31, which can effectively buffer the impact force when the hatching eggs are placed, avoid damage to the hatching eggs due to collision, and achieve damage-free support for the hatching eggs. Four electric rollers 33 are rotatably installed between the two buffer elements 32, and the four electric rollers 33 are connected end to end.

[0030] The rotation of the electric roller 33 can drive the hatching eggs to rotate smoothly, thereby realizing flexible adjustment of the hatching egg angle. Compared with the traditional manual adjustment method, it is not only more efficient, but also more accurate in angle adjustment, which can meet the needs of the sensing unit to collect information on different angles of the hatching eggs.

[0031] The damper 321 can be welded to the upper side surface of the bracket 31. The two ends of the spring 322, which is sleeved on the telescopic part of the damper 321, are respectively welded to the telescopic part of the damper 321 and the housing, forming a double buffer structure of "damper + spring". When the hatching egg is placed on the electric roller 33, this structure can effectively absorb the pressure brought by the weight of the hatching egg and the vibration generated when the electric roller 33 rotates through the elastic deformation of the spring 322 and the damping effect of the damper 321. On the one hand, it can prevent the hatching egg from being damaged by excessive pressure or vibration, ensuring the safety of the hatching egg; on the other hand, it can reduce the impact of vibration on the posture of the hatching egg, ensuring that the hatching egg remains stable during the detection process and improving the accuracy of the information collected by the sensing unit.

[0032] Specifically, the electric roller 33 includes a motor roller 331, a shaft 332, and a rubber sleeve 333. The shaft 332 is the output shaft of the damper 321, and the shafts 332 of the four electric rollers 33 are connected end to end. The shafts 332 at both ends are rotatably connected to two dampers 321 respectively, so that the optimal fit can be adjusted according to the shape and weight of the hatching eggs. The rubber sleeve 333 is glued to the outer surface of the motor roller 331.

[0033] The shaft 332 serves as the output shaft of the damper 321, and the shafts 332 of the four electric rollers 33 are connected end to end, with both ends rotatably connected to the two dampers 321. This connection method ensures the synchronous rotation of the four electric rollers 33, avoiding the deviation or jamming of hatching eggs due to inconsistent rotation speeds of individual rollers, and improving the stability of hatching egg angle adjustment. The rubber sleeve 333 bonded to the outer surface of the motor roller 331 has good elasticity and anti-slip properties, which can increase the friction between the hatching egg and the electric roller 33, preventing the hatching egg from slipping during rotation, and can also buffer the contact pressure between the hatching egg and the roller through the elastic deformation of the rubber sleeve 333, avoiding damage to the hatching egg.

[0034] The two egg supports are symmetrically arranged to provide stable support for the eggs and have an angle adjustment function, which can adjust the eggs to the optimal detection posture. This ensures that the sensing unit can collect egg information from all directions without blind spots, solving the problem of incomplete information collection caused by the fixed posture of the eggs in traditional detection.

[0035] The sensing unit includes an infrared laser thermometer 11, a transmission light source 4, and a multi-angle camera. As the core component for information acquisition, the sensing unit can accurately capture multi-dimensional status information of hatching eggs.

[0036] In some embodiments, the transmission camera is mounted directly above the hatching egg position; the multi-angle camera includes a first camera 8 and a second camera 13; the first camera 8 is mounted diagonally above the hatching egg position; the second camera 13 is mounted to the side of the hatching egg position.

[0037] The infrared laser thermometer 11 is located on the side of the hatching eggs and is mounted by a clamping component 10. The clamping component 10 is mounted on the third electric telescopic rod 9, which is fixed to the box on the side of the weighing platform. Figure 1 and 2 As shown.

[0038] The third electric telescopic rod 9 can flexibly adjust the position of the clamping component 10 and the infrared laser thermometer 11. On the one hand, it can adjust the infrared laser thermometer 11 to the optimal temperature measurement position according to the size and placement angle of the hatching eggs, ensuring the accuracy of the temperature measurement point and avoiding temperature data distortion caused by temperature position deviation. On the other hand, it can reset the infrared laser thermometer 11 after the test is completed, facilitating the handling of hatching eggs. The infrared laser thermometer 11 is positioned on the right side of the hatching eggs, complementing the multi-angle acquisition of the multi-view camera without interfering with image acquisition. At the same time, it can quickly and non-destructively acquire the temperature data of the hatching eggs. This temperature data can be combined with image data for multimodal fusion analysis, further improving the accuracy of hatching egg sex detection.

[0039] like Figure 6 and 7 As shown, the clamping component 10 includes an iron plate 101 welded to the telescopic end of the third electric telescopic rod 9. A rubber pad 102 is bonded to the upper surface of the iron plate 101. An anti-slip groove 103 for limiting the infrared laser thermometer 11 is opened on the upper surface of the rubber pad 102. A rubber strip 104 is provided above the rubber pad 102. A magnet block 105 is bonded to the holes opened at both ends of the rubber strip 104. A slot 106 for limiting the magnet block 105 is opened on the side of the iron plate 101, and the iron plate 101 and the magnet block 105 are magnetically fixed together.

[0040] The adhesive pad 102 bonded to the upper surface of the iron plate 101 has a cushioning and anti-slip function. The anti-slip groove 103 on the adhesive pad 102 can limit the infrared laser thermometer 11, preventing the infrared laser thermometer 11 from shifting due to equipment vibration during the detection process, ensuring the consistency of the temperature measurement point. At the same time, the adhesive pad 102 can reduce the impact of vibration on the infrared laser thermometer 11 and ensure its working stability. The magnet blocks 105 at both ends of the adhesive strip 104 are magnetically fixed to the slots 106 on the side of the iron plate 101. This magnetic fixing method can not only firmly press the infrared laser thermometer 11 into the anti-slip groove 103 to prevent it from falling off, but also facilitate the disassembly, assembly, maintenance and calibration of the infrared laser thermometer 11. Compared with the traditional bolt fixing method, the operation is more convenient and efficient, improving the maintenance convenience of the equipment.

[0041] The transmitted light source 4 is located directly below the hatching egg. The transmitted light source 4 can be a 10W green lamp with a wavelength of 520nm. Under the transmission of this light source, the blood lines of the embryo in the hatching egg can be clearly seen. Figure 8 The image shown is of the bloodline in the hatching egg. This allows for even lighting of the egg from below, preventing shadows and enhancing the light transmittance of the egg's internal structure, making it easier for a transmission camera to capture internal details.

[0042] The detection system of this invention constructs a multimodal information acquisition system through a sensing unit composed of a transmission light source 4, a multi-angle camera, and an infrared laser thermometer 11, providing comprehensive data support for accurate sex determination of hatching eggs. The transmission light source 4 ensures sufficient and uniform light during the detection process, avoiding blurry images and loss of detail caused by insufficient or uneven light distribution. The multi-angle camera captures image information such as the appearance and internal structure of the hatching eggs, providing a basis for sex determination based on image features. The infrared laser thermometer 11 collects temperature data from the hatching eggs, using the differences in temperature distribution between male and female eggs to assist in sex determination. The fusion of multimodal information acquisition overcomes the limitations of traditional single-information detection methods, enabling the mining of sex characteristics from multiple dimensions and significantly improving the accuracy and reliability of sex detection.

[0043] In addition, the support mechanism, through a symmetrical egg support frame, combined with a "damper + spring" buffer structure, and multiple series-connected electric rollers for shape adjustment, can achieve non-destructive and stable support for the eggs, and can also achieve precise and automated adjustment of the egg angle through the electric rollers. This avoids the vibration interference factors when adjusting the angle in existing methods, thus making the test results more accurate.

[0044] The image acquisition system includes a multi-angle camera and a transmission camera 6 (which may be a CCD microscope), a first electric telescopic rod 5, a second electric telescopic rod 7, a first camera 8, a fourth electric telescopic rod 12, and a second camera 13. The first electric telescopic rod 5 is fixed to the inner surface of the top of the detection box 1 with screws. The CCD microscope is installed at the telescopic end of the first electric telescopic rod 5 and is located directly above the hatching egg. The second electric telescopic rod 7 is fixed at an angle to the inner surface of the middle of the detection box 1 with screws. The first camera 8 is installed at the telescopic end of the second electric telescopic rod 7 and is located diagonally above the hatching egg. The fourth electric telescopic rod 12 is fixed to the inner surface of the bottom of the detection box 1 with screws. The second camera 13 is installed at the telescopic end of the fourth electric telescopic rod 12 and is located to the left of the hatching egg.

[0045] The first motorized telescopic rod 5 moves the CCD microscope vertically above the hatching egg. The CCD microscope, with its high resolution, can clearly capture the fine structural features inside the egg, providing high-definition microscopic image data for accurate sex determination. The motorized telescopic rod's design allows for flexible adjustment of the CCD microscope's detection distance according to the egg size, improving the adaptability of the detection. The second motorized telescopic rod 7 moves the first camera 8 diagonally above the egg, acquiring macroscopic image information from the side of the egg, supplementing the blind spot of the top-down detection. The fourth motorized telescopic rod 12 moves the second camera 13 to the left side of the egg, acquiring image information from the left side of the egg, further improving the image acquisition dimensions. This multi-camera, multi-view layout, combined with the flexible adjustment function of the motorized telescopic rods, ensures that image information from all parts of the egg can be accurately acquired, providing a rich and comprehensive image data source for subsequent multimodal data fusion analysis, effectively improving the accuracy of sex determination.

[0046] In some embodiments, the system further includes an interaction device 14, which is mounted on the side surface of the detection box 1 and is electrically connected to the weighing platform 2, the egg support frame, and the sensor unit.

[0047] The electrical connection design between the interactive device 14 and the weighing platform 2, the egg support frame, and the sensing unit enables centralized control and data interaction of each component. Operators can start the testing process, adjust parameters, and view data through the interactive device 14, which greatly improves the convenience and automation of the testing operation and reduces the intensity of manual operation. The interactive device can be set with a touch screen for operation.

[0048] In this embodiment, the transmission light source 4 installed on the weighing part of the weighing platform 2 is located directly below the hatching egg, which can provide uniform illumination to the hatching egg from below, avoid shadows appearing below the hatching egg, and at the same time enhance the light transmittance of the internal structure of the hatching egg, making it easier for the CCD microscope to capture the internal details of the hatching egg.

[0049] In some embodiments, the transmission image and the infrared image are images of the bleeding stage, and the weighing platform 2 is used to acquire first weight information at the start of hatching and second weight information at the bleeding stage; the detection module determines the density loss of the hatching egg at the bleeding stage based on the volume information and in combination with the first weight information and the second weight information; correspondingly, the detection module determines the sex type of the hatching egg based on the density loss, the transmission image, and the infrared image.

[0050] Furthermore, in some embodiments, the detection module determines the sex type of the hatching egg based on a sex detection model. The sex detection model includes an image segmentation network, a numerical extraction network, a feature fusion network, and a classification prediction network. Specifically: the image segmentation network is used to input a transmission image of the bloodline stage and output a first image feature; the numerical extraction network is used to input density loss and two-dimensional infrared image data and output a second image feature; the feature fusion network is used to fuse the first image feature and the second image feature and input them into the classification prediction network to obtain the sex detection result of the egg; wherein the sex detection model is trained based on sample hatching eggs with determined sex results.

[0051] In some embodiments, the bloodline period includes the fourth day of egg incubation, as shown in Figure 8.

[0052] In some embodiments, the image segmentation network includes a U-Net or ResNet-50 module, and the numerical extraction network includes a fully connected neural network. In some embodiments, a Self-Attention mechanism is incorporated into the gender detection model.

[0053] Using day 4 of incubation, earlier than the sensitive period of embryonic development, as the key detection node, a standardized process was used to collect three-modal data: egg weight, multi-directional embryonic bloodline images, and multi-site temperature data. After preprocessing, a deep learning architecture of "single-modal feature extraction + mid-term fusion" was used to complete sex determination. The entire process was achieved without damaging the eggs, with a quantifiable detection process and traceable results. The specific steps are as follows: Image data optimization: Preprocessing of acquired embryonic bloodline images (transmission images), air cell images, and whole egg images: ① Gaussian filtering algorithm is used for noise reduction to eliminate image interference; ② Histogram equalization is used for image enhancement to improve the clarity of bloodline details; ③ Region of interest (ROI) is cropped to retain key areas such as embryonic bloodline and air cell, and invalid background is removed; ④ 3D reconstruction algorithm is optimized to remove abnormal pixels in the image and improve the accuracy of predicting the true volume of hatching eggs.

[0054] Dataset construction: The preprocessed image data and numerical data are associated with gender labels by unique IDs to construct a structured multimodal dataset, which is divided into training set, validation set and test set in a ratio of 14:3:3.

[0055] The deep learning model is constructed using a core architecture of "single-modal feature extraction + mid-term fusion", and the model training and optimization are completed by the algorithm engine mounted on the data processing module.

[0056] 1. First image feature extraction: Embryonic bloodline image feature extraction: U-Net or ResNet-50 model is selected. The preprocessed bloodline image is input, and deep features are extracted through convolutional layers and pooling layers. The output is a 256-dimensional or 512-dimensional image feature vector. U-Net is good at capturing fine texture features of bloodlines, while ResNet-50 can reduce gradient vanishing and improve the ability to extract deep features.

[0057] 2. Second Image Feature Extraction: For example, the infrared image has dimensions H×W. The density loss (which can be normalized first) is a single value, such as 0.5. During stitching, the density loss can be expanded into an H×W×1 feature map matching the spatial dimensions of the infrared image (with the same value at each spatial location), and then stitched with the infrared image along the channel dimension. The second image features are then obtained through a feature extraction network, such as a 64-dimensional numerical feature vector, which is used for subsequent fusion with the first image features.

[0058] 3. Multimodal mid-term fusion: The 512-dimensional first image features and the 64-dimensional numerical features are concatenated to obtain a 576-dimensional fused feature vector; a Self-Attention mechanism is added to allow the model to automatically allocate the weights of each modality feature, such as giving priority to core information such as blood line features, making full use of complementary information between modalities to improve prediction accuracy. Compared with early fusion, it can retain deep effective features, and compared with late fusion, it can enhance modal synergy.

[0059] 4. Classification Prediction and Training Optimization: The fused feature vector is input into the fully connected classification layer and mapped to a 2D male / female output. The class probability is calculated using the Softmax function. The cross-entropy loss function is used to adapt to the binary classification task. The Adam optimizer is used with an initial learning rate of 1e-4. Dropout probability of 0.3 and L2 regularization are added to avoid overfitting. Accuracy is used as the core indicator, and the model performance is comprehensively evaluated by combining recall, precision, and F1 score. The training parameters such as learning rate and number of iterations are dynamically adjusted based on the validation set until the model performance is stable.

[0060] The obtained stable model is stored in the detection module as a gender detection model for subsequent detection.

[0061] To verify the detection effect of density loss measurement and the fusion of infrared image (temperature distribution) and transmission image, 850 healthy hatching eggs (day 4 of incubation, bloodline stage) of the same Jingfen No. 1 breed were selected as the experimental sample. Under the same detection environment and equipment parameters, four feature combination methods were used to conduct sex detection experiments. The experimental models all adopted the "single-modal feature extraction + mid-term fusion" deep learning architecture proposed in this paper. The sex result verified manually after hatching was used as the gold standard. The detection accuracy, precision, and F1 score of each scheme were statistically analyzed. The specific experimental data are shown in the table below: Feature combination method Detection accuracy Accuracy F1 score Single transmission image (control group) 86.2% 85.7% 85.9% Transmission image + density loss 88.5% 88.1% 88.3% Transmission image + infrared image (temperature distribution) 89.1% 88.8% 88.9% Transmission image + density loss measurement + infrared image (Solution of this application) 92.8% 92.6% 92.7% The experimental results clearly show that the accuracy rate is only 86.2% when using transmission image detection alone. After adding density loss and infrared image features to the transmission image, the detection accuracy is significantly improved. When using the three-feature fusion scheme of transmission image + density loss + infrared image proposed in this application, the detection accuracy, precision, and F1 score are all improved to over 92%, which is 6.6 percentage points higher than the single transmission image scheme. This fully demonstrates that the density loss and temperature distribution data corresponding to the infrared image, as feature inputs, can effectively complement the transmission image. Through multimodal feature fusion, the accuracy of egg sex detection is significantly improved, solving the technical problems of weak anti-interference ability and limited accuracy of single transmission image detection.

[0062] In some embodiments, the two buffers 32 of each egg support are tilted upward and inward.

[0063] In some embodiments, the rubber sleeve 333 is provided with annular buffer cavities 334 at both ends.

[0064] In some embodiments, the side surface of the rubber sleeve 333 is a concave rotational surface.

[0065] The buffer cavities 334 at both ends of the rubber sleeve 333 can limit and buffer the two ends of the hatching egg, preventing the hatching egg from moving in the axial direction. At the same time, during the placement or rotation of the hatching egg, the buffer cavities 334 can absorb the impact force at the ends, further ensuring the safety of the hatching egg. The structure of the rubber sleeve 33 and buffer cavity 334 of the electric roller 33 further ensures that the hatching egg is undamaged and rotates smoothly, avoiding slippage and displacement problems during inspection.

[0066] The embodiments described above are merely illustrative, and some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement these embodiments without any creative effort.

[0067] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all fall within the protection scope of the appended claims of the present invention.

Claims

1. A system for detecting the sex of hatching eggs based on multimodal information fusion, characterized in that, It includes a testing box, weighing platform, support mechanism, transmission light source, transmission camera, infrared laser thermometer, multi-angle camera, and testing module; The testing box is a closed cavity structure with an openable and closable door; The support mechanism is fixedly installed on the top of the weighing platform and is used to adjust the posture of the hatching eggs placed on it. The support mechanism includes a pair of hatching egg support frames. Each hatching egg support frame includes a pair of brackets, a pair of buffers installed on the corresponding brackets, and a rotating component disposed between the pair of buffers. The buffers include a damper and a spring. The damper includes a housing and a telescopic part. One end of the housing is fixedly connected to the bracket. The telescopic part is slidably connected to the housing and extends out of the other end of the housing. One end of the spring is connected to the housing, and the other end is connected to the telescopic part. The rotating component includes multiple electric rollers connected in series. Each electric roller includes a motor roller, a shaft, and a rubber sleeve. The end of the shaft is hinged to the shaft of the telescopic part or the adjacent electric roller. The motor roller is rotatably installed outside the shaft and drives the rubber sleeve to rotate. The weighing platform is installed inside the testing box and is used to obtain the weight information of the hatching eggs; The multi-angle camera is used to obtain the volume information of the hatching eggs based on three-dimensional reconstruction; The infrared laser thermometer is located on the side of the hatching egg and is used to acquire infrared images of the hatching egg; The transmission camera is used to acquire a transmission image of the hatching egg under transmission light source illumination; The detection module is used to determine the sex type of the hatching egg based on the weight information, the volume information, the transmission image, and the infrared image.

2. The egg sex detection system based on multimodal information fusion according to claim 1, characterized in that, The transmission camera is installed directly above the location of the hatching egg; The multi-angle camera includes a first camera and a second camera; The first camera is installed diagonally above the location of the hatching egg; The second camera is mounted on the side of the egg location.

3. The egg sex detection system based on multimodal information fusion according to claim 1, characterized in that, The transmitted image and the infrared image are images of the bloodline stage, and the weighing platform is used to obtain the first weight information of the hatching eggs at the beginning of incubation and the second weight information of incubation to the bloodline stage. The detection module determines the density loss of the hatching egg from the bloodline stage based on the volume information, combined with the first weight information and the second weight information. Accordingly, the detection module determines the sex type of the hatching egg based on the density loss, the transmission image, and the infrared image.

4. The egg sex detection system based on multimodal information fusion according to claim 3, characterized in that, The detection module determines the sex type of the hatching eggs based on a sex detection model, which includes an image segmentation network, a numerical extraction network, a feature fusion network, and a classification prediction network, wherein: The image segmentation network is used as input for a transmissive image of the vascular phase and outputs a first image feature; The numerical extraction network is used to take density loss and two-dimensional infrared image data as input, and output second image features; The feature fusion network is used to fuse the first image features and the second image features, and input them into the classification prediction network to obtain the sex detection result of the poultry egg; The sex detection model is trained based on sample eggs whose sex has been determined.

5. The egg sex detection system based on multimodal information fusion according to claim 3 or 4, wherein the bloodline period includes the fourth day of egg incubation.

6. The egg sex detection system based on multimodal information fusion according to claim 4, characterized in that, The image segmentation network includes a U-Net or ResNet-50 module, and the numerical extraction network includes a fully connected neural network.

7. The egg sex detection system based on multimodal information fusion according to claim 4, characterized in that, The gender detection model incorporates a Self-Attention mechanism.

8. The egg sex detection system based on multimodal information fusion according to claim 1, characterized in that, The two buffers of each egg support frame are tilted upwards and inwards.

9. The egg sex detection system based on multimodal information fusion according to claim 1, characterized in that, The rubber sleeve has annular buffer cavities at both ends.

10. The egg sex detection system based on multimodal information fusion according to claim 1, characterized in that, The side of the rubber sleeve is a concave rotating surface.

Images