Intelligent identification and counting system for food microbial colonies based on deep learning

Abstract

The present application belongs to the technical field of food microorganism detection, and discloses a food microorganism colony intelligent identification and counting system based on deep learning; image data of different illumination angles of a culture dish is acquired, a meniscus transition zone is accurately identified, and the culture surface is divided into a central flat area and an edge meniscus area. A three-dimensional meniscus surface model is constructed based on photometric stereo method, a refraction offset is calculated to realize distortion correction, and brightness normalization is realized through Fresnel reflection analysis. An adaptive segmentation strategy is adopted for different areas, a position-aware attention mechanism based on radial distance coding is introduced, and the recognition effect is optimized by dynamically adjusting the lighting parameters to accurately process the cross-area colonies. The present application realizes high-precision colony identification and counting in the whole area of the culture dish, and improves the integrity and reliability of microorganism detection.

Description

Technical Field

[0001] This invention relates to the field of food microbial detection technology, and more specifically, to a deep learning-based intelligent identification and counting system for food microbial colonies. Background Technology

[0002] Traditional microbial colony counting systems face severe edge recognition challenges in practical applications, primarily due to the meniscus effect formed by the culture medium at the edge of the petri dish and the resulting complex optical phenomena. Due to surface tension between the liquid culture medium and the dish wall, the edge region inevitably forms an upward-curving liquid surface, creating an optical structure similar to a concave lens. This curved surface structure causes severe light refraction and reflection anomalies, resulting in nonlinear geometric distortions in the edge colony images. The morphology is distorted and stretched, the size proportions are distorted, and in extreme cases, a mirror-flip effect may occur. Simultaneously, the enhanced Fresnel reflection caused by the curved surface leads to localized overexposure or irregular light spots in the edge region, while changes in the surface angle create localized shadow areas, resulting in a significant difference in contrast between the edge colonies and the central region. In routine laboratory operations, especially during high-throughput screening, these edge regions often become "identification blind spots," causing systematic counting bias. Traditional algorithms, employing globally consistent processing parameters, cannot adapt to the special optical characteristics of the edge regions, while partitioned processing strategies cannot effectively handle cross-regional colonies, creating new sources of error. With increasingly stringent food safety standards, identification errors in these edge regions are no longer acceptable, especially in the analysis of low-contamination samples. Missing edge colonies can lead to false negatives, posing potential food safety risks. The meniscus region typically occupies a significant proportion of the petri dish area, and the accuracy of identification in this region directly impacts the overall reliability of the system.

[0003] In view of this, the present invention proposes a deep learning-based intelligent identification and counting system for food microbial colonies to solve the above problems. Summary of the Invention

[0004] To overcome the aforementioned deficiencies of the prior art and to achieve the above objectives, the present invention provides the following technical solution: a deep learning-based intelligent identification and counting system for food microbial colonies, comprising:

[0005] The multi-angle acquisition module is used to acquire multi-angle illumination image data of the culture surface of the petri dish. It acquires images under different illumination azimuth and elevation angles to form a multi-angle illumination image set.

[0006] The region segmentation module is used to detect the circular boundary of the culture dish based on a multi-angle illumination image set, calculate the radial gray-level gradient distribution curve, identify the meniscus transition zone based on the gradient abrupt change position in the radial gray-level gradient distribution curve, and divide the culture surface into a central flat region and an edge meniscus region.

[0007] The surface reconstruction module is used to utilize the difference in reflectance brightness of the meniscus region under different illumination angles, estimate the surface normal vector of each pixel based on the photometric stereo method, reconstruct the height field of the culture medium liquid surface by integrating the normal vector, and construct a three-dimensional surface model of the meniscus.

[0008] The geometric distortion correction module is used to calculate the pixel offset after the imaging light is refracted by the curved liquid surface based on the local surface normal vector of each pixel position in the three-dimensional curved surface model of the meniscus, generate a geometric distortion correction vector field, perform inverse mapping correction on the image of the edge meniscus region, and generate a distortion-corrected image.

[0009] The light intensity compensation module is used to calculate the incident light angle and Fresnel reflection coefficient of each surface micro-element based on the three-dimensional curved surface model of the meniscus, determine the theoretical transmitted light intensity of each pixel in the edge meniscus region, generate an intensity compensation coefficient map, and perform brightness normalization processing on the distortion-corrected image.

[0010] The candidate region extraction module is used to calculate the local adaptive segmentation threshold for the central flat region and the edge crescent region after brightness normalization, extract the colony candidate region, calculate the area, roundness, gray-level contrast and edge gradient magnitude of each colony candidate region, and construct the morphological feature vector.

[0011] The classification and recognition module is used to input the local image patches and morphological feature vectors of each colony candidate region into the colony classification network. The colony classification network includes an edge position awareness attention module based on radial distance coding, which outputs the colony category and recognition confidence of each colony candidate region.

[0012] The result verification and output module is used to mark the candidate colony regions with recognition confidence scores below the preset confidence threshold in the edge crescent region as regions to be verified based on the spatial distribution of recognition confidence scores. The module adjusts the lighting parameters of the regions to be verified and re-acquires images to update the recognition results and outputs the colony location, category and total count.

[0013] The modules are connected via wired and / or wireless means to enable data transmission between them.

[0014] The technical effects and advantages of the deep learning-based intelligent identification and counting system for food microbial colonies in this invention are as follows:

[0015] This invention improves the fidelity of colony morphology in edge regions through multi-dimensional optical reconstruction and deep feature learning, ensuring consistent feature expression with the central region and eliminating edge blind spots in traditional techniques. The position-aware mechanism of this invention adaptively processes the characteristics of different regions of the culture dish, effectively overcoming morphological distortion and shadow interference caused by curved surfaces. The dynamic optical optimization strategy enables the system to cope with complex lighting conditions, improving imaging quality and feature extraction effects in difficult-to-identify areas. The cross-regional consistency analysis mechanism solves the problem of colony attribution at regional boundaries, eliminating the risk of duplicate counting and undercounting. These comprehensive improvements work together to provide consistent analytical performance across the entire culture dish, enhancing the completeness and reliability of microbial detection. The comprehensive colony information output by this invention supports in-depth microbiological analysis and meets the stringent requirements for accurate quantification of microorganisms. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the intelligent identification and counting system for food microbial colonies based on deep learning, as described in this invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] This application provides a deep learning-based intelligent identification and counting system for food microbial colonies. The system's execution entities include, but are not limited to, food safety testing platforms, microbiological analysis experimental platforms, intelligent laboratory management systems, and food quality control centers, which can be considered general computing nodes in this application. The identification and counting system includes, but is not limited to, at least one deep learning colony analysis engine, a distributed image processing system, and an intelligent petri dish scanner.

[0019] Please see Figure 1 In this embodiment of the invention, the food microbial colony intelligent identification and counting system based on deep learning includes:

[0020] The multi-angle acquisition module is used to acquire multi-angle illumination image data of the culture surface of petri dishes. Images are acquired under different illumination azimuth and elevation angles to form a multi-angle illumination image set. The multi-angle illumination image data includes petri dish images under different combinations of azimuth and elevation angles, acquired synchronously through multiple sets of light sources distributed in a ring. Different illumination azimuth angles reflect the light distribution of the petri dish in the horizontal plane, while different elevation angles provide illumination variations in the vertical direction, together constituting omnidirectional illumination conditions. This multi-angle illumination data provides rich illumination information for subsequent surface reconstruction, ensuring accurate colony identification under different illumination conditions and improving the robustness of the system in complex illumination environments.

[0021] The region segmentation module detects the circular boundary of the culture dish based on a multi-angle illumination image set, calculates the radial gray-level gradient distribution curve, and identifies the meniscus transition zone based on the gradient abrupt change locations in the radial gray-level gradient distribution curve, dividing the culture surface into a central flat region and an edge meniscus region. The central flat region is the approximately horizontal central part of the culture medium surface, with relatively stable optical properties; the edge meniscus region is the curved liquid surface area near the culture dish wall, where the complex refraction and reflection of light due to changes in surface curvature creates a unique crescent-shaped appearance. This region segmentation strategy allows for differentiated processing of the optical properties of different regions, improving overall recognition accuracy.

[0022] The surface reconstruction module utilizes the difference in reflected brightness of the meniscus region under different illumination angles to estimate the surface normal vector of each pixel based on the photometric stereo method. The height field of the culture medium surface is reconstructed through normal vector integration, constructing a 3D surface model of the meniscus. This 3D surface model accurately describes the geometry of the culture medium surface, providing a mathematical basis for subsequent geometric distortion correction and light intensity compensation. By analyzing brightness variation patterns under multi-angle illumination, this model inversely reproduces the surface morphology, avoiding the limitation of traditional methods requiring additional depth sensors and achieving purely vision-based 3D reconstruction.

[0023] The geometric distortion correction module calculates the pixel offset after the imaging light rays are refracted by the curved liquid surface based on the local surface normal vectors of each pixel position in the meniscus 3D surface model. It then generates a geometric distortion correction vector field, performs inverse mapping correction on the image of the meniscus edge region, and generates a distortion-corrected image. The geometric distortion correction vector field describes the offset relationship between the actual position and the imaging position of each pixel, restoring the distorted image to its normal shape through inverse mapping. This correction process solves the problem of colony morphology deformation caused by refraction in the edge region, providing a morphologically accurate image foundation for subsequent colony identification.

[0024] The light intensity compensation module calculates the incident light angle and Fresnel reflection coefficient of each surface element based on the 3D surface model of the meniscus, determines the theoretical transmitted light intensity of each pixel in the meniscus region, generates an intensity compensation coefficient map, and performs brightness normalization processing on the distortion-corrected image. The intensity compensation coefficient map quantifies the light intensity change of each pixel due to the surface angle, and by multiplying by the corresponding compensation coefficient, achieves brightness uniformity across the entire culture dish. This processing solves the overexposure problem caused by enhanced reflection in the edge region, ensuring that the colonies have similar grayscale characteristics throughout the entire culture dish, thus providing conditions for a unified segmentation algorithm.

[0025] The candidate region extraction module calculates local adaptive segmentation thresholds for the central flat region and the edge crescent region after brightness normalization, extracting candidate colony regions. It then calculates the area, roundness, grayscale contrast, and edge gradient magnitude of each candidate colony region to construct a morphological feature vector. This morphological feature vector comprehensively describes the geometric and grayscale characteristics of the colony, providing multi-dimensional feature support for subsequent classification and recognition. By using adaptive thresholds with different window sizes to distinguish different regions, the system can adapt to changes in local illumination and background, improving the accuracy of candidate region extraction.

[0026] The classification and recognition module inputs local image patches and morphological feature vectors of each colony candidate region into the colony classification network. This network includes an edge-position-aware attention module based on radial distance encoding, outputting the colony category and recognition confidence score for each candidate region. The colony classification network integrates visual and morphological features and considers the positional information of colonies in the petri dish through a specially designed attention mechanism, achieving high-precision colony classification. This network can distinguish different types of microbial colonies and provides a confidence score for each recognition result, providing a basis for subsequent result verification.

[0027] The results verification and output module, based on the spatial distribution of recognition confidence, marks candidate colony regions with recognition confidence below a preset confidence threshold within the meniscus region as areas to be verified. For these areas, illumination parameters are adjusted, images are reacquired, and the recognition results are updated. The module then outputs the colony location, category, and total count. By specifically adjusting the illumination parameters in low-confidence regions, the system can adaptively improve the imaging quality of challenging areas and enhance recognition accuracy. The final output not only includes the total number of colonies but also provides the precise location and category information for each colony, meeting the detailed needs of microbial analysis.

[0028] The modules are connected via wired and / or wireless means to enable data transmission between them.

[0029] In this embodiment of the invention, the detailed implementation steps for acquiring multi-angle illumination image data of the culture surface of a petri dish, and acquiring images under different illumination azimuth and elevation angles to form a multi-angle illumination image set, include:

[0030] Multiple light sources are arranged in a ring above the petri dish, each located at a different azimuth angle. This ring-shaped distribution ensures that the light source covers all directions of the petri dish, typically using 8-12 evenly distributed azimuth angles, covering a range from 0° to 360°. Each azimuth angle light source group forms an independently controllable illumination unit, allowing for precise control of the light direction and intensity. Narrow-band LEDs are used to ensure spectral consistency and reduce the impact of color difference. Precise azimuth angle positioning is controlled by a high-precision stepper motor, ensuring an angle error of less than 0.5°, providing an accurate angular reference for subsequent photometric stereoscopic analysis. This ring-shaped distribution design not only provides omnidirectional illumination but also allows for the simulation of various complex lighting environments through combinations of different light sources.

[0031] Each light source group comprises multiple emitting units arranged at different elevation angles. Each unit is illuminated sequentially, and a frame of image is acquired synchronously. The elevation angle distribution typically covers a range of 15° to 75°, with multiple emitting units spaced at 5° or 10° intervals. Lower elevation angles (nearly horizontal) help highlight subtle surface undulations, while higher elevation angles (nearly vertical) provide more uniform overall illumination. Each emitting unit is equipped with an independent drive circuit and brightness controller, with the illumination sequence precisely controlled by a microprocessor. During image acquisition, a high-speed synchronization controller ensures precise synchronization between the emitting unit illumination and camera exposure, with time errors controlled at the microsecond level. Only one emitting unit is illuminated at a time, acquiring the corresponding frame of image. By cyclically illuminating all emitting units, a complete multi-angle illumination dataset is obtained, providing sufficient information on illumination variations for subsequent 3D reconstruction.

[0032] Dark-field correction and flat-field normalization are performed on all images to eliminate inherent sensor noise differences. Dark-field correction removes sensor thermal noise and fixed-pattern noise by subtracting the dark-field image acquired under no-light conditions; flat-field normalization corrects lens vignetting and sensor pixel response inconsistencies by dividing by the response of a uniform white board under the same lighting conditions. The mathematical representation of the correction process is as follows:

[0033] ;

[0034] in, For the corrected image, For the original image, This is a dark field image. A flat-field image. This represents the average grayscale value of the flat field image.

[0035] The calibration process is accelerated by dedicated image processing hardware, ensuring real-time processing capabilities. The system automatically acquires dark-field and flat-field images periodically to adapt to environmental changes and maintain calibration accuracy. This calibration not only improves the signal-to-noise ratio of the images but also ensures consistent grayscale response for images acquired at different locations and times, providing a reliable data foundation for subsequent analysis.

[0036] The corrected images are indexed and arranged according to their azimuth and elevation angles to form a multi-angle illumination image set. The indexing uses a multi-dimensional array structure, with the first dimension representing the azimuth index and the second dimension representing the elevation index, facilitating rapid access to images under specific illumination conditions. The system establishes an illumination metadata table, recording the precise light source location, intensity, and spectral characteristics corresponding to each image, providing accurate reference data for subsequent photometric analysis. The image set is stored in an efficient hierarchical storage format, supporting fast random access and batch processing. The system also implements a preprocessing caching mechanism for the image set, loading frequently accessed images into memory or GPU memory to accelerate subsequent analysis and processing. As the system's fundamental data resource, the multi-angle illumination image set provides comprehensive visual information for subsequent steps such as region segmentation and surface reconstruction.

[0037] In this embodiment of the invention, the detailed implementation steps for detecting the circular boundary of a culture dish based on a multi-angle illumination image set, calculating the radial gray-level gradient distribution curve, identifying the meniscus transition zone based on the gradient abrupt change location in the radial gray-level gradient distribution curve, and dividing the culture surface into a central flat region and an edge meniscus region include:

[0038] A circular Hough transform is performed on the mean image of all images in a multi-angle illumination image set to detect the outer boundary circle of the culture dish. The mean image is obtained by calculating the pixel average of the images under all illumination conditions, eliminating shadow and highlight interference from single illumination conditions and highlighting the stable structural features of the culture dish. The circular Hough transform is a classic algorithm for detecting circular boundaries, identifying circular structures in the image through a parameter space voting mechanism. To improve detection accuracy, the system first performs adaptive binarization and edge extraction preprocessing on the mean image, and then applies a multi-scale Hough transform to search for the optimal circular boundary within different radius ranges. The algorithm sets appropriate accumulator thresholds and edge point ratio thresholds to exclude the influence of interfering circles and incomplete circles. The detection result returns the center coordinates (x0, y0) and radius r of the outer boundary circle of the culture dish, providing a reference coordinate system for subsequent radial analysis.

[0039] Starting from the center of the outer boundary circle, grayscale values ​​are sampled along multiple radial rays. The radial gradient of grayscale at each sampling point is calculated, and the median of the gradients along each ray is used to form a radial grayscale gradient distribution curve. The radial rays are uniformly distributed within the range of 0° to 360°, typically using 72-360 rays to ensure sufficient angular coverage. Grayscale values ​​are sampled along each ray from the center outwards in pixel increments, forming a one-dimensional grayscale profile. The radial grayscale gradient is calculated using the central difference method.

[0040] ;

[0041] in, Radial position grayscale gradient at that location This represents the grayscale value at the corresponding position. This is the sampling step size (usually 1 pixel).

[0042] To suppress the influence of noise, the system first applies Gaussian smoothing to the gradient curve of each ray, and then takes the median of the gradient values ​​of all ray directions at the same radial distance to form the final radial gray-scale gradient distribution curve. The median operation effectively resists the interference of local outliers and colony edges, extracting the gradient characteristics of the culture dish structure itself.

[0043] The radial position where the absolute value of the gradient first exceeds a preset gradient threshold in the radial gray-scale gradient distribution curve is identified and defined as the inner boundary of the meniscus transition zone. The meniscus transition zone is the region where the culture medium surface transitions from flat to curved, exhibiting a significant gradient change on the gradient curve. The detection process employs an adaptive threshold method, with the preset gradient threshold typically set to 15%-25% of the global maximum value of the gradient curve to accommodate different culture medium concentrations and lighting conditions. The system searches outwards along the gradient curve from the center to find the first position exceeding the threshold, marking the boundary between the flat and meniscus regions. To improve detection stability, the system implements a multi-scale validation mechanism, repeating the detection at different smoothing scales and using the median of the results as the final boundary position, effectively resisting noise and local anomaly interference.

[0044] The region within the inner boundary of the meniscus transition zone is defined as the central flat region, and the annular region between the inner boundary of the meniscus transition zone and the outer boundary of the culture dish is defined as the edge meniscus region. The central flat region is approximately a horizontal plane with relatively simple optical properties, allowing for direct application of traditional image processing methods. The edge meniscus region, however, requires special processing to compensate for optical distortion and reflection changes caused by the curved surface. The system generates a region mask map, dividing the culture dish region into the central flat region (pixel value 255), the edge meniscus region (pixel value 128), and the background region (pixel value 0), facilitating the application of differentiated processing strategies for different regions in subsequent modules. Simultaneously, the system calculates and stores statistical characteristics such as the area ratio, average gray level, and variance of the two regions, providing a basis for subsequent adaptive parameter adjustments. The region division results intuitively reflect the structural characteristics of the culture dish, laying the foundation for accurate colony identification in the meniscus region.

[0045] In this embodiment of the invention, the detailed implementation steps for constructing a three-dimensional surface model of the meniscus include: utilizing the difference in reflectance brightness of the meniscus region at different illumination angles, estimating the surface normal vector of each pixel based on the photometric stereo method, reconstructing the height field of the culture medium liquid surface through normal vector integration, and constructing the model are as follows:

[0046] The grayscale values ​​of the same pixel within the meniscus region are extracted from images at different illumination angles to form the photometric sampling vector for that pixel. The photometric sampling vector reflects the reflection characteristics of surface micro-elements under different illumination directions and is the fundamental data for estimating the surface normal vector. The extraction process first uses a meniscus mask generated by the region segmentation module to filter out the pixels to be processed; then, for each pixel, the grayscale values ​​at that location under different illumination conditions are extracted from a multi-angle illumination image set to form a high-dimensional photometric vector. To improve the signal-to-noise ratio, the system performs a Gaussian weighted average on the local region (usually a 3×3 or 5×5 window) for each illumination condition to reduce the impact of single-pixel noise. The dimension of the photometric sampling vector is consistent with the number of illumination conditions, typically ranging from tens to hundreds of dimensions, containing rich surface reflection information.

[0047] Based on the Lambertian reflection assumption, the surface normal vector of a pixel is calculated from the photometric sampling vector using the least squares method. The Lambertian reflection model assumes that the surface reflection intensity is proportional to the cosine of the light source direction and the surface normal vector, making it suitable for describing the optical properties of diffuse reflective materials such as culture media. According to Lambertian's law of reflection, the reflection intensity... It can be represented as:

[0048] ;

[0049] in, The diffuse reflectance coefficient is... For the surface normal vector, Let be the direction vector of the light source. This represents the vector dot product operation.

[0050] For multi-source conditions, an overdetermined system of equations is formed:

[0051] ;in, For the first The direction vector of each light source This represents the total number of light sources.

[0052] The system solves the system of equations using the least squares method to obtain the optimal normal vector estimate:

[0053] ;

[0054] in, Here is the light source direction matrix. The observed intensity vector is solved and normalized to obtain the unit normal vector. The system incorporates robust estimation techniques, employing iterative reweighted least squares to suppress outlier effects and improve the accuracy of normal vector estimation.

[0055] Median filtering is applied to the obtained surface normal vector field to suppress noise in the normal vector estimation caused by the semi-transparent nature of the culture medium. The semi-transparency of the culture medium causes some light to penetrate the surface and scatter internally, violating the assumptions of the Lambertian reflection model and generating local estimation noise. Median filtering is an effective method to preserve edge characteristics while suppressing noise. , , Median filtering is applied to each of the three components to preserve the structural characteristics of the normal vector field. The filtering window size is adaptively adjusted according to the noise level, typically ranging from 5×5 to 9×9 pixels. After filtering, each normal vector is renormalized to a unit vector to ensure physical consistency. To further improve smoothness, the system also employs bilateral filtering technology to suppress noise while preserving the main features of the surface structure, forming a coherent normal vector field.

[0056] Based on the filtered normal vector field, the liquid surface height field of the meniscus region is reconstructed by integrating the Poisson equation. The transformation from the normal vector field to the height field is a crucial step in surface reconstruction, achieved by solving partial differential equations. A relationship exists between the surface normal vector and the height gradient:

[0057] ;

[0058] Transformed into a height gradient field:

[0059] ;

[0060] Ideally, the gradient field should satisfy the integrability condition: However, due to measurement errors, the actual gradient field usually does not satisfy this condition. The system solves this by solving the Poisson equation:

[0061] ;

[0062] The optimal height field estimate is obtained. The Fast Fourier Transform (FFT) method is employed, which is highly efficient and globally optimal. During integration, the height of the central flat region is set as the reference zero point to ensure the continuity and physical plausibility of the height field.

[0063] In the formula, for The Laplace operator, In spatial location The surface unit normal vector at that location, , and The normal vectors are respectively in , and Components in direction; and The height field is respectively in and Gradient in the direction.

[0064] A parametric surface fitting method is used to fit the liquid surface height field, generating a 3D meniscus surface model. The parametric model provides a compact representation of the height field, facilitating subsequent distortion correction and light intensity compensation calculations. The fitting process employs a piecewise bicubic spline surface model, balancing representational capability and computational efficiency. The system first divides the height field into a uniform grid, fitting a local bicubic surface on each grid, and then ensuring the continuity of surfaces between adjacent grids through smoothing constraints. The fitting optimization uses the Levenberg-Marquardt algorithm to minimize the squared difference between the model height and the measured height, while incorporating a smoothness regularization term to prevent overfitting. The final generated 3D meniscus surface model is represented by a coefficient matrix, supporting rapid calculation of height and normal vectors at arbitrary locations, providing an accurate geometric basis for subsequent distortion correction and light intensity compensation. Model evaluation is performed by calculating the fitting error and surface smoothness index to ensure the reconstruction quality meets the requirements of subsequent processing.

[0065] In this embodiment of the invention, based on the local surface normal vectors of each pixel position in the three-dimensional surface model of the meniscus, the pixel offset after the imaging light is refracted by the curved liquid surface is calculated, a geometric distortion correction vector field is generated, and inverse mapping correction is performed on the image of the edge meniscus region to generate a distortion-corrected image. The detailed implementation steps include:

[0066] The surface normal vector and surface height value at each pixel position are extracted from the 3D surface model of the meniscus. Mapping the pixel position to the corresponding point on the 3D surface is the first step in distortion correction. The system establishes the correspondence between the pixel coordinate system and the 3D spatial coordinate system, projecting each pixel onto the 3D surface model. For each pixel in the meniscus region at the edge... The system calculates the corresponding three-dimensional spatial coordinates from the parametric surface model. , where z is the height value; simultaneously calculate the local surface normal vector at that point. This refers to the unit normal vector of the surface at that point. The calculation process utilizes the analytical expression of the surface model and employs bicubic interpolation for points within the mesh to ensure smooth continuity of the coordinates and normal vectors. The system also calculates the principal curvatures and principal directions of the surface as supplementary descriptions of the surface's geometric properties, providing more comprehensive information for subsequent optical analysis.

[0067] Based on the refractive indices of the culture medium and air, Snell's law is applied to the imaging rays passing through each pixel position to calculate the refraction deflection angle. Refraction is the main physical mechanism causing distortion in edge regions, and Snell's law can be used to accurately calculate the light path deflection. The culture medium, as a transparent medium, typically has a refractive index between 1.33 and 1.38, varying slightly depending on its specific composition; the refractive index of air is approximately 1.0. The system is configured with the camera optical axis perpendicular to the plane of the culture dish, establishing a light propagation model. For rays emanating from inside the culture medium, the angle between the ray and the normal vector is first calculated. Then, Snell's law is applied to calculate the angle after refraction. :

[0068] ;

[0069] in, The refractive index of the culture medium, Let be the refractive index of air. The system considers refraction in three-dimensional space, projecting the incident light ray and normal vector onto a plane containing the camera's optical axis, and calculating the refraction deflection angles in the horizontal and vertical directions respectively, achieving comprehensive three-dimensional optical path tracing. The refraction calculation takes into account the spectral dispersion characteristics of the culture medium, using corresponding refractive index values ​​for different wavelengths to eliminate the influence of chromatic aberration.

[0070] By combining the surface height value and the refraction deflection angle, the lateral offset of each pixel on the imaging plane is calculated, forming a geometric distortion correction vector field. The lateral offset reflects the imaging position shift caused by refraction and is the core parameter for distortion correction. The calculation process is based on geometric optics principles, mapping the ray tracing results onto the imaging plane. For a surface with a height of... The point, the angle of refraction is lateral offset The calculation is as follows:

[0071] ;

[0072] Considering the imaging characteristics of actual cameras, the system also introduces a camera intrinsic parameter matrix to convert the physical space offset into image pixel offset:

[0073] ;

[0074] in, and This represents the horizontal offset in the pixel coordinate system. and For the camera's focal length parameter, and In physical space and The offset in direction. The system organizes the offsets of all pixels into a vector field, namely the distortion correction vector field, which is used to guide subsequent image resampling.

[0075] An inverse mapping lookup table is established based on the geometric distortion correction vector field. The image is then resampled in the meniscus region using bilinear interpolation to generate a distortion-corrected image. Inverse mapping is an effective method for distortion correction, avoiding the holes and overlap problems that may occur with forward mapping. The process of establishing the inverse mapping lookup table involves: for each pixel position of the corrected image... The corresponding position of the distortion correction vector field in the original distorted image is found. Since the calculated corresponding positions are usually not integer coordinates, the system uses bilinear interpolation to interpolate from the surrounding four integer coordinate pixels to obtain an accurate grayscale value. For color images, interpolation is performed separately for the RGB channels. The resampling process is accelerated by the GPU, supporting real-time processing of large images. The correction result preserves the resolution and visual information of the original image, while correcting the geometric distortion caused by surface refraction, allowing the colonies in the meniscus region to exhibit their true morphological characteristics, providing accurate morphological information for subsequent colony identification.

[0076] In this embodiment of the invention, the detailed implementation steps for calculating the incident light angle and Fresnel reflection coefficient of each surface micro-element based on the three-dimensional curved surface model of the meniscus, determining the theoretical transmitted light intensity of each pixel in the meniscus region, generating an intensity compensation coefficient map, and performing brightness normalization processing on the distortion-corrected image include:

[0077] Based on the three-dimensional surface model of the meniscus and the position of the light source, the angle between the incident ray and the surface normal vector at each surface micro-element is calculated. The incident angle is a key parameter determining the surface's reflectivity, directly affecting the ratio of light energy reflection to transmission. The calculation process first establishes a unified three-dimensional coordinate system, representing the surface model and the light source position in the same reference frame; then, for each point on the surface, the direction vector from that point to the illumination source is calculated. and the surface normal vector at that point Calculate the included angle:

[0078] ;

[0079] in, The incident angle is given. Considering that multiple light sources typically illuminate simultaneously in practical applications, the system calculates the contribution of each light source and performs a weighted average based on the light source intensity to obtain the comprehensive incident angle. For each pixel in the crescent region, the system stores its corresponding incident angle information, forming an incident angle distribution map, which provides basic data for subsequent reflection coefficient calculations.

[0080] Based on the incident angle and the refractive index of the culture medium, the reflection coefficient at each surface micro-element is calculated using the Fresnel equation, and thus the transmission coefficient is obtained. The Fresnel equation accurately describes the reflection and transmission behavior of light at the interface of the medium and is the theoretical basis for calculating the light intensity distribution. For a given incident angle... Reflectance coefficient Calculated as the average of the parallel and vertical polarization components:

[0081] ;

[0082] in, and The refractive indices of air and culture medium are respectively. The transmission angle is calculated using Snell's law:

[0083] ;

[0084] Transmission coefficient Then, according to the principle of conservation of energy:

[0085] ;

[0086] The system takes into account the bilayer structure of the culture medium (air-culture medium-bottom of the petri dish) and calculates the combined effect of multiple reflections to more accurately simulate the actual optical path. The calculation results form a reflectance coefficient map and a transmittance coefficient map, which intuitively show the influence of the curved surface shape on light energy transmission. Steep edge regions usually exhibit higher reflectance and lower transmittance.

[0087] The average gray value of the central flat region is used as the reference brightness, and the ratio of the reference brightness to the theoretical transmittance coefficient of each pixel is used as the intensity compensation coefficient for that pixel, forming an intensity compensation coefficient map. Because the surface of the central flat region is approximately horizontal, its reflection and transmission characteristics are relatively stable, and its average gray value can be used as the brightness reference for the entire culture dish. The system first calculates the average gray value of the central flat region. Then, for each pixel in the meniscus region, based on its theoretical transmission coefficient... Calculate the strength compensation coefficient :

[0088] ;

[0089] This calculation is based on the assumption that, under ideal uniform illumination conditions, the gray value of a pixel in an image is proportional to the light energy passing through that point, while the transmittance coefficient... This describes the proportion of light energy loss due to reflection from curved surfaces. Multiplying by... The compensation coefficient can restore the brightness of the edge area to the same level as the center area, eliminating brightness unevenness caused by changes in surface curvature. The intensity compensation coefficient map generated by the system visually shows the compensation intensity required for different areas; steep edge areas usually require a higher compensation coefficient.

[0090] The grayscale values ​​of each pixel in the meniscus region of the distortion-corrected image are multiplied by the corresponding intensity compensation coefficient to perform brightness normalization. Brightness normalization is a crucial step in achieving grayscale consistency across the entire culture dish region, laying the foundation for subsequent unified threshold segmentation. For the distortion-corrected image... The system is based on the strength compensation coefficient diagram. Perform pixel-level multiplication operations to generate a brightness-normalized image. :

[0091] ;in, This represents the pixel position.

[0092] To prevent overcompensation from causing pixel saturation, the system implements an adaptive limiting strategy to ensure that the compensated grayscale values ​​do not exceed the effective dynamic range. For color images, the system adjusts the luminance component while maintaining hue consistency to preserve natural color transitions. The image after luminance normalization exhibits consistent grayscale characteristics across the entire region, with colonies in the meniscus region at the edges showing similar contrast and luminance distribution to those in the central region, thus facilitating subsequent unified feature extraction and colony identification.

[0093] In this embodiment of the invention, the detailed implementation steps for calculating local adaptive segmentation thresholds for the central flat region and the edge crescent region after brightness normalization, extracting colony candidate regions, and constructing morphological feature vectors include:

[0094] For the central flat region, the local mean and local standard deviation are calculated using a first window size to determine the adaptive segmentation threshold for the central flat region. The background of the central flat region is relatively uniform, making it suitable for using a larger window for local statistical analysis. The first window size is typically set to 31×31 to 51×51 pixels, which can cover sufficient background information while adapting to local variations. The system employs a sliding window method, calculating the mean within a local window centered on each pixel location in the image. and standard deviation To improve computational efficiency, the system uses integral graph technology to reduce the computational complexity of window statistics from... Reduce to It supports real-time processing of large windows. Adaptive thresholding. The calculation formula is:

[0095] ;

[0096] in, The threshold factor is typically set to 2-3 and automatically adjusted based on the contrast between colonies and the background. The threshold calculation takes into account local background variations, adapting to uneven lighting and changes in culture medium concentration, thus improving the robustness of segmentation. The system generates a threshold map of the central region, visually displaying the spatial distribution of the segmentation thresholds and providing accurate local thresholds for subsequent binarization segmentation.

[0097] For the meniscus region at the edge, a second window size smaller than the first window is used to calculate the local mean and local standard deviation, determining the adaptive segmentation threshold for the meniscus region. The background variations in the meniscus region are more complex, requiring a smaller window to accommodate local characteristics. The second window size is typically set to 15×15 to 25×25 pixels to capture local variations more precisely. The reason for using a smaller window is that, despite geometric distortion correction and brightness normalization, the background in the edge region may still exhibit high-frequency variations and subtle noise, requiring a more localized threshold to adapt to these variations. The calculation process is similar to that in the center region, but the parameter settings differ; the threshold factor k is typically slightly smaller to accommodate the potentially lower contrast in the edge region. The system introduces a directional window technique to address the special properties of the edge region. The window shape is stretched along the main direction of the curved surface to better adapt to the radial characteristics of the edge region, improving the accuracy of the threshold calculation.

[0098] The system uses corresponding adaptive segmentation thresholds to perform binarization segmentation on the central flat region and the edge crescent region, respectively, and extracts connected regions as colony candidate regions. Binarization segmentation is a key step in converting the grayscale image into two categories: colonies and background. Based on the previously calculated local adaptive thresholds, the system classifies each pixel in the image:

[0099] ;

[0100] in, For the binarized result, For input grayscale images, An adaptive threshold is used. After segmentation, the system applies morphological opening operations to remove small noise points, and then performs connected component analysis to label regions composed of all connected foreground pixels. To handle colony overlap, the system implements a watershed-based region segmentation algorithm, which can effectively separate slightly overlapping colonies. The system filters out regions with excessively small areas (typically less than 5-10 pixels) to eliminate noise interference, while retaining all possible candidate colony regions, including small colonies. Each candidate region is assigned a unique identifier for subsequent feature extraction and classification.

[0101] For each candidate colony region, the system calculates the number of pixels in area, the number of pixels in perimeter, the difference in mean grayscale values ​​between the inside and outside of the region, and the mean gradient value of the boundary pixels, combining these to form a morphological feature vector. This morphological feature vector comprehensively describes the geometric and grayscale characteristics of the colony and is an important basis for colony classification. The system calculates the following features for each candidate region:

[0102] 1) Area pixels: The total number of pixels in the area, reflecting the size of the colony;

[0103] 2) Perimeter pixels: The number of pixels at the region boundary, calculated using 8-connected chain code technology to accurately measure the boundary length;

[0104] 3) Roundness: Use the formula The area is calculated as × area / perimeter², which measures how close the shape of the area is to a circle. The closer the value is to 1, the more round it is.

[0105] 4) Difference in mean grayscale between the area and the surrounding background: The difference in grayscale between the colony area and the surrounding background, calculated using the formula | Inside- outside, among which The value within the region is the mean. The outer region is the average background value within a certain range outside the region (usually 1 / 4 of the region width);

[0106] 5) Mean gradient of boundary pixels: The mean grayscale gradient magnitude calculated along the region boundary. The gradient is calculated using the Sobel operator and reflects the clarity of the boundary.

[0107] The system combines these features into feature vectors for each candidate region, and performs dimensionality reduction using Principal Component Analysis (PCA) or Linear Discriminant Analysis (LDA) to highlight the most discriminative feature combinations. The morphological feature vectors comprehensively characterize the visible properties of the colonies, providing high-quality feature input for subsequent deep learning classification and improving the accuracy and efficiency of classification.

[0108] In this embodiment of the invention, the detailed implementation steps of the colony classification network including an edge location-aware attention module based on radial distance encoding include:

[0109] The radial distance from the centroid of each candidate colony region to the center of the culture dish is calculated, and the radial distance is normalized and encoded as a position embedding vector. Radial distance encoding is a key step in fusing positional information, enabling the network to perceive the relative position of colonies within the culture dish. The calculation process first determines the centroid coordinates of each candidate region (…). , ), calculate its distance from the center of the petri dish ( , Euclidean distance:

[0110] ;

[0111] Then divide the distance by the radius of the petri dish. Normalization is performed to obtain the relative radial distance. The value range is [0,1]. To enable the network to better utilize location information, the system converts the scalar distance into a high-dimensional embedding vector and adopts a sine-cosine position coding method:

[0112] ;

[0113] ;

[0114] in, For embedding dimension indexes, The total embedding dimension is typically 16-64. This encoding method preserves the continuity and monotonicity of distance while mapping location information to a high-dimensional space, facilitating the network's learning of complex location-related features. The system generates a location embedding vector for each candidate region, which serves as the basis for subsequent feature modulation.

[0115] The location embedding vector is multiplied channel-wise with the feature map extracted from the local image patch of the colony candidate region through convolution to generate a location-modulated feature map. Location modulation is the core mechanism for achieving location awareness, enabling a sensitive response to location information by modulating visual features. The processing flow first extracts a fixed-size local image patch (usually 64×64 or 128×128 pixels) for each candidate region, ensuring that it contains the complete colony and its surrounding background; then, a multi-channel feature map F is extracted through a convolutional neural network (such as the first few layers of ResNet or EfficientNet); simultaneously, the location embedding vector is multiplied channel-wise. The modulation factor is converted to match the number of channels in the feature map through a fully connected layer and a reshape operation. Finally, channel-level multiplication is performed to obtain the position modulation feature map. :

[0116] ;

[0117] in, For channel indexing, The first feature map after position modulation Each channel is a feature representation modulated by location information. The value of the c-th channel in the position modulation factor is a modulation coefficient derived from the position embedding vector through a fully connected layer and a reshape operation. This modulation method allows the network to adaptively adjust the importance of features based on the colony's location, for example, by giving specific feature enhancements to colonies in edge regions, thus improving recognition accuracy. The position modulation feature map combines visual appearance and positional information, providing a rich information foundation for subsequent attention mechanisms.

[0118] An attention gating layer is set before the classification head of the colony classification network. This layer generates channel attention weights based on the position modulation feature map, weighting the feature responses of different channels. The attention mechanism is an effective strategy to improve network performance, achieving more accurate feature extraction by adaptively adjusting feature importance. The attention gating layer adopts a squeeze-and-excitation architecture. First, global average pooling is performed on the position modulation feature map, compressing the spatial dimension into channel descriptors. Then, two fully connected layers generate channel attention weights. The first layer reduces the dimensionality (typically to 1 / 16 of the number of channels) and uses ReLU activation, while the second layer restores the dimensionality and uses Sigmoid activation, ensuring the weights are within the range [0,1]. Finally, the channel attention weights are multiplied by the input features to achieve feature weighting.

[0119] ;

[0120] ;

[0121] ;

[0122] in, For the first Channel descriptors for each channel, The width of the feature map. The height of the feature map, For the position modulation feature map, the first The spatial location of each channel eigenvalues ​​at that location This is a vector containing all channel descriptors. and For the weights of the fully connected layer, For the Sigmoid function, For attention weights, The first one after attention weighting through the channel Channel characteristics, The first in the position modulation feature map Each channel has an attention mechanism that adaptively adjusts the importance of features in each channel based on the location characteristics of the colony. For example, it enhances texture features for colonies at the edges and shape features for colonies in the center, thereby improving the network's accuracy in recognizing colonies at different locations.

[0123] A colony classification network was trained using a training set containing samples of centrally flat regions and edge crescent regions after geometric distortion correction and brightness normalization. The quality and diversity of the training data are crucial for network performance. The training set construction process included: first, collecting a large number of petri dish images of different microorganisms to ensure coverage of common colony types and morphological variations; then, assigning accurate category labels and bounding boxes to each colony region through expert annotation; and finally, applying data augmentation techniques, including rotation, scaling, brightness variations, and noise addition, to expand the training samples and improve the model's generalization ability. The training process employed the cross-entropy loss function and the Adam optimizer, with an initial learning rate of 0.001 and a learning rate decay strategy. To prevent overfitting, weight decay and Dropout regularization were introduced, along with an early stopping strategy, stopping training when the validation set performance no longer improved. Through multiple rounds of iterative optimization, the network learned the combined expression of visual and positional features of colonies, forming a unified recognition capability for colonies across the entire petri dish. Once trained, the model can accurately identify various types of colonies at different locations and provides a confidence score for each identification result, supporting subsequent result verification and optimization.

[0124] In this embodiment of the invention, the detailed implementation steps for adjusting the illumination parameters of the area to be reviewed and re-acquiring the image to update the recognition result include:

[0125] The system analyzes the recognition confidence and corresponding morphological feature vectors of each candidate colony region in the area to be verified to determine the dominant factors causing low confidence. The area to be verified is typically a difficult-to-identify meniscus region, requiring detailed analysis of the reasons for recognition uncertainty. The system uses multidimensional analysis to determine the dominant factors causing low confidence: first, it examines the recognition confidence distribution to identify regions significantly below the average level; then, it analyzes the morphological feature vectors of these regions to detect abnormal feature dimensions; finally, by comparing the features with those of high-confidence regions, it identifies the main factors causing recognition difficulties. The system focuses on two main types of problems: insufficient grayscale contrast (small difference in grayscale between colonies and background) and blurred edge gradients (unclear colony boundaries). Grayscale contrast is evaluated by the difference between the mean grayscale values ​​inside and outside the region; when the difference is lower than a preset contrast threshold (usually 2-3 times the background standard deviation), it is considered insufficient contrast. Edge gradient is evaluated by the mean gradient of boundary pixels; when the mean is lower than a preset gradient threshold, it is considered blurred edges. Based on the analysis results, the system marks each area to be reviewed as having insufficient contrast or blurred edges, providing targeted guidance for subsequent adjustments to lighting parameters.

[0126] When the dominant factor is grayscale contrast below a preset contrast threshold, increasing the light source elevation angle reduces the angle between the incident light direction and the surface normal vector of the area, thus increasing the positive illumination intensity of the area. Insufficient contrast is usually caused by an inappropriate light incident angle; proper adjustment can significantly improve the imaging effect. The system calculates the average normal vector of the area to be verified based on the surface model, then selects the light source position closest to the direction of this normal vector, and increases the elevation angle of the light source at that position, making the incident light more parallel to the surface normal vector. This adjustment is based on the physical principle that when light is nearly perpendicularly incident, surface reflection decreases and transmission increases, increasing the light flux through the culture medium and enhancing the contrast between colonies and the background. The system typically increases the elevation angle by 10°-20°, while simultaneously increasing the brightness of the light source by 10%-30% to ensure sufficient illumination intensity. For particularly difficult areas, the system will also activate two light sources at symmetrical positions to illuminate simultaneously, forming a dual-light source illumination mode to further improve contrast. This dynamic illumination adjustment can specifically improve the imaging quality of specific areas and increase the accuracy of colony identification.

[0127] When the dominant factor is that the edge gradient amplitude is lower than the preset gradient threshold, the system switches to lateral illumination mode to enhance the contrast between light and dark at the colony edges. Blurred edges are often caused by overly uniform lighting; lateral illumination can produce directional shadows, enhancing edge features. The system first determines the azimuth angle of the area to be verified relative to the center of the culture dish, then selects a light source position perpendicular to that azimuth angle, and sets a low elevation angle (typically 20°-30°) with high intensity lateral illumination. This illumination configuration makes the light almost parallel to the culture medium surface, producing obvious shadows and highlights when encountering height changes at the colony edges, emphasizing the edge contours. To further enhance the effect, the system simultaneously turns off light sources in other directions, increasing the directional contrast of the illumination. For non-circular colonies, the system may require lateral illumination from multiple directions. By rapidly switching between light sources in different directions and synthesizing multiple frames of images, the system ensures that edge features from all directions are captured. Lateral illumination technology is particularly suitable for identifying colonies with blurred boundaries in the meniscus region, significantly improving the quality of edge feature extraction.

[0128] Based on the adjusted illumination parameters, images of the area to be verified are reacquired, and geometric distortion correction, brightness normalization, and colony classification are re-performed. The recognition confidence score is updated until it exceeds a preset confidence threshold or reaches the preset maximum number of verifications. Reacquisition and processing is an iterative process to optimize the recognition results. The system controls the light source to reacquire images of the area to be verified based on the adjusted illumination parameters, typically capturing only a portion of the petri dish to improve processing efficiency. After acquiring new images, the system re-performs geometric distortion correction and brightness normalization according to the established process, and then inputs them into the colony classification network for re-identification. The system compares the updated recognition confidence score with the preset threshold (usually 0.85-0.95). If it exceeds the threshold, the current result is accepted; if it is still below the threshold and the maximum number of verifications (usually 3-5 times) has not been reached, the system will further adjust the illumination parameters based on the new feature analysis results and begin the next round of verification. To prevent infinite loops, when the preset maximum number of verifications is reached, the system selects the result with the highest confidence score among all verification results as the final result. This adaptive iterative optimization mechanism can make multiple attempts for areas that are difficult to identify, significantly improving the overall recognition accuracy of the system, especially for colony recognition in the crescent-shaped edge region.

[0129] In this embodiment of the invention, the output of colony location, category, and total count further includes:

[0130] The system detects cross-regional colony candidate regions located on the boundary between the central flat region and the edge meniscus. This boundary region serves as a transition zone between two processing strategies, potentially leading to duplicate counting or undercounting of the same colony, requiring special handling. The system first determines the inner boundary curve of the edge meniscus, forming a ring-shaped buffer zone with a width equal to the average colony diameter, covering the possible transition area. Then, all colony candidate regions are retrieved, and colonies whose central or partial areas fall within the buffer zone are identified and marked as cross-region candidate regions. To ensure detection integrity, the system simultaneously checks the segmentation results of both regions, merging connected regions that may have been separated during the region division process. For colonies with complex shapes or large sizes, the system applies morphological dilation to temporarily expand the candidate region's range, improving the sensitivity of overlap detection. This buffer strategy and dual-region detection mechanism ensure that colonies near the boundary line are not incorrectly processed due to region division, improving the accuracy of the overall count.

[0131] For cross-regional colony candidate regions, morphological feature vectors are extracted from the central flat region and the peripheral crescent region. The two parts of a cross-regional colony may exhibit different characteristics due to different processing methods, requiring separate analysis and comprehensive judgment. The system first uses a region mask to divide the cross-regional colony candidate region into a central part and an edge part, calculating the morphological feature vectors of each part separately, including key features such as area, roundness, grayscale contrast, and edge gradient value. To ensure the accuracy of feature extraction, the system applies specific processing parameters to each part; the central part uses the parameters set for the central region, and the edge part uses the parameters set for the edge region. Feature extraction considers the possibility that some regions may have small areas; feature scaling and standardization ensure the comparability of the features of the two parts. The system also calculates the differences in morphological changes between the two parts in the original and processed images, providing additional evidence for subsequent consistency analysis. This differentiated processing ensures accurate feature description of cross-regional colonies, providing a reliable data foundation for determining whether they belong to the same colony.

[0132] The system calculates the feature consistency metric between two morphological feature vectors. When the feature consistency metric is greater than a preset consistency threshold, the two parts are merged into a single colony; when the feature consistency metric is less than or equal to the preset consistency threshold, they are determined to be two independent colonies. Feature consistency is a key indicator for determining whether two regions belong to the same colony. The system uses weighted cosine similarity to calculate the similarity of feature vectors, with the following formula:

[0133] ;

[0134] in, for and Weighted cosine similarity between them and These are the two parts of the feature vector. The weights for each feature dimension reflect the importance of the feature; For feature vectors The Each component value For feature vectors The Each component value This represents the total dimension of the feature vector. Weight settings typically emphasize shape features (such as roundness) and texture features (such as grayscale contrast distribution), while reducing the influence of location-related features. The preset consistency threshold is usually set between 0.75 and 0.85, determined through optimization using a validation dataset. For boundary cases, the system also considers supplementary factors such as spatial continuity and regional overlap, applying a voting mechanism for final determination. When the consistency metric is higher than the threshold, the system merges the two parts into a single colony, using an area-weighted average to determine the final feature vector and center position; when the consistency metric is lower than or equal to the threshold, they are retained as two independent colonies, counted separately in the total count. This feature similarity-based determination mechanism effectively solves the problem of duplicate counting of cross-regional colonies, improving the accuracy of the counting results.

[0135] The merged or split colonies are included in the final count. The final count is a key output of the system, requiring the integration of identification information from all regions and ensuring accuracy. The system first aggregates all candidate colony regions in the central flat area and the meniscus area, eliminating false positives caused by low confidence and significant noise. Then, cross-regional colonies are processed, merging or retaining them based on the aforementioned consistency analysis results, and updating the total colony list. For each confirmed colony, the system records its precise location (x, y coordinates in the petri dish coordinate system), size (equivalent diameter), morphological characteristics (roundness, density, etc.), and classification result (colony category and confidence level). The system also calculates the quantitative statistics and spatial distribution characteristics of various colonies, such as density heatmaps and aggregation analysis, providing more comprehensive colony distribution information. The final output includes: total colony count, classification statistics, detailed information table for each colony, and visualization results (original chart with colony location and category labeled). The system supports multiple output formats, including CSV data tables, JSON structured data, and PDF reports, facilitating integration with laboratory information management systems and subsequent analysis. This comprehensive and detailed output meets the professional needs of microbiological analysis, providing accurate counting results and a wealth of auxiliary information.

[0136] This invention achieves accurate identification and counting of bacterial colonies across the entire culture dish by employing multi-angle acquisition, region segmentation, surface reconstruction, geometric distortion correction, light intensity compensation, candidate region extraction, classification and recognition, and result verification and output. The meniscus surface processing method of this invention effectively solves the problem of identifying bacterial colonies in edge regions, the edge position-aware attention mechanism improves the accuracy of identifying colonies at different locations, and the adaptive illumination parameter adjustment mechanism optimizes the identification effect in difficult areas.

[0137] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0138] It should be noted that all formulas in this manual are calculated by removing dimensions and taking their numerical values. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters and thresholds in the formulas are set by those skilled in the art according to the actual situation.

[0139] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. A deep learning-based intelligent identification and counting system for food microbial colonies, characterized in that, include: The multi-angle acquisition module is used to acquire multi-angle illumination image data of the culture surface of the petri dish. It acquires images under different illumination azimuth and elevation angles to form a multi-angle illumination image set. The region segmentation module is used to detect the circular boundary of the culture dish based on the multi-angle illumination image set, calculate the radial gray-level gradient distribution curve, identify the meniscus transition zone according to the gradient abrupt change position in the radial gray-level gradient distribution curve, and divide the culture surface into a central flat region and an edge meniscus region. The surface reconstruction module is used to estimate the surface normal vector of each pixel based on the photometric stereo method by utilizing the difference in reflectance brightness of the meniscus region under different illumination angles, and to reconstruct the height field of the culture medium liquid surface by integrating the normal vectors, thereby constructing a three-dimensional surface model of the meniscus. The geometric distortion correction module is used to calculate the pixel offset after the imaging light is refracted by the curved liquid surface based on the local surface normal vector of each pixel position in the three-dimensional curved surface model of the meniscus, generate a geometric distortion correction vector field, perform inverse mapping correction on the image of the edge meniscus region, and generate a distortion correction image. The light intensity compensation module is used to calculate the incident light angle and Fresnel reflection coefficient of each surface micro-element based on the three-dimensional curved surface model of the meniscus, determine the theoretical transmitted light intensity of each pixel in the edge meniscus region, generate an intensity compensation coefficient map, and perform brightness normalization processing on the distortion correction image. The candidate region extraction module is used to calculate local adaptive segmentation thresholds for the central flat region and the edge crescent region after brightness normalization, extract colony candidate regions, calculate the area, roundness, grayscale contrast and edge gradient magnitude of each colony candidate region, and construct morphological feature vectors. The classification and recognition module is used to input the local image patches of each of the colony candidate regions and the morphological feature vectors into the colony classification network. The colony classification network includes an edge position awareness attention module based on radial distance coding and outputs the colony category and recognition confidence of each of the colony candidate regions. The result verification and output module is used to mark the candidate colony regions with identification confidence scores lower than a preset confidence threshold in the meniscus region as regions to be verified based on the spatial distribution of the identification confidence scores. The module then adjusts the lighting parameters of the regions to be verified, re-acquires images, updates the identification results, and outputs the colony location, category, and total count.

2. The system according to claim 1, characterized in that, The acquisition of multi-angle illumination image data of the culture surface of the petri dish involves acquiring images under different illumination azimuth and elevation angles to form a multi-angle illumination image set, including: Multiple sets of light sources are arranged in a ring above the petri dish, with each set of light sources located at a different azimuth angle. Each light source group contains multiple light-emitting units arranged at different elevation angles. Each light-emitting unit is lit up one by one and an image frame is captured synchronously. Perform dark field correction and flat field normalization on all images; The corrected images are indexed and arranged according to their azimuth and elevation angles to form the multi-angle illumination image set.

3. The system according to claim 1, characterized in that, The process involves detecting the circular boundary of the culture dish based on the multi-angle illumination image set, calculating the radial gray-level gradient distribution curve, identifying the meniscus transition zone based on the gradient abrupt change location in the radial gray-level gradient distribution curve, and dividing the culture surface into a central flat region and an edge meniscus region, including: Perform a circular Hough transform on the mean image of all images in the multi-angle illumination image set to detect the outer boundary circle of the culture dish; Starting from the center of the outer boundary circle, gray values ​​are sampled along multiple radial ray directions, the gray radial gradient of each sampling point is calculated, and the median of the gradients in each ray direction is taken to form the radial gray gradient distribution curve. The radial position where the absolute value of the gradient first exceeds a preset gradient threshold is detected in the radial gray-scale gradient distribution curve, and this position is determined as the inner boundary of the meniscus transition zone. The area within the inner boundary of the meniscus transition zone is defined as the central flat area, and the annular area between the inner boundary of the meniscus transition zone and the outer boundary of the culture dish is defined as the edge meniscus area.

4. The system according to claim 1, characterized in that, The method utilizes the difference in reflected brightness of the meniscus region under different illumination angles, estimates the surface normal vector of each pixel based on the photometric stereo method, reconstructs the culture medium liquid level height field through normal vector integration, and constructs a three-dimensional surface model of the meniscus, including: Extract the grayscale values ​​of the same pixel within the meniscus region of the edge in images at different illumination angles to form the photometric sampling vector of that pixel; Based on the Lambertian reflection hypothesis, the surface normal vector of the pixel is solved by the least squares method from the photometric sampling vector. Median filtering is applied to the obtained surface normal vector field; Based on the filtered normal vector field, the liquid surface height field of the edge meniscus region is reconstructed by integrating the Poisson equation. Parametric surface fitting is performed on the liquid level height field to generate a three-dimensional surface model of the meniscus.

5. The system according to claim 1, characterized in that, The step involves calculating the pixel offset of the imaging light after refraction by the curved liquid surface based on the local surface normal vector of each pixel position in the three-dimensional surface model of the meniscus, generating a geometric distortion correction vector field, and performing inverse mapping correction on the image of the edge meniscus region to generate a distortion-corrected image, including: Extract the surface normal vector and surface height value at each pixel position from the three-dimensional surface model of the meniscus; Based on the refractive index of the culture medium and the refractive index of air, the refraction deflection angle is calculated by applying Snell's law to the imaging light rays passing through each pixel position. By combining the surface height value and the refraction deflection angle, the lateral offset of each pixel on the imaging plane is calculated to form the geometric distortion correction vector field. An inverse mapping lookup table is established based on the geometric distortion correction vector field, and the image of the edge crescent region is resampled by bilinear interpolation to generate the distortion correction image.

6. The system according to claim 1, characterized in that, The process of calculating the incident light angle and Fresnel reflection coefficient of each surface micro-element based on the three-dimensional curved surface model of the meniscus, determining the theoretical transmitted light intensity of each pixel in the meniscus region, generating an intensity compensation coefficient map, and performing brightness normalization processing on the distortion-corrected image includes: Based on the three-dimensional surface model of the meniscus and the position of the light source, calculate the angle between the incident light ray and the surface normal vector at each surface micro-element; Based on the included angle and the refractive index of the culture medium, the reflection coefficient at each surface micro-element is calculated using the Fresnel equation, and then the transmission coefficient is obtained. The average gray value of the central flat area is used as the reference brightness, and the ratio of the reference brightness to the theoretical transmission coefficient of each pixel is used as the intensity compensation coefficient of that pixel, thus forming the intensity compensation coefficient map. The brightness normalization process is completed by multiplying the gray value of each pixel in the meniscus region of the distortion-corrected image by the corresponding intensity compensation coefficient.

7. The system according to claim 1, characterized in that, The process involves calculating local adaptive segmentation thresholds for the brightness-normalized central flat region and the edge crescent region, extracting candidate colony regions, and constructing morphological feature vectors, including: For the central flat region, calculate the local mean and local standard deviation using a first window size, and determine the adaptive segmentation threshold for the central flat region; For the meniscus region at the edge, a second window size smaller than the first window size is used to calculate the local mean and local standard deviation, and an adaptive segmentation threshold for the meniscus region at the edge is determined. The central flat region and the edge crescent region are binarized using their respective adaptive segmentation thresholds, and the connected regions are extracted as the colony candidate regions. For each candidate colony region, the number of pixels in area, the number of pixels in perimeter, the mean difference in gray levels inside and outside the region, and the mean gradient of boundary pixels are calculated and combined to form the morphological feature vector.

8. The system according to claim 1, characterized in that, The colony classification network includes an edge location-aware attention module based on radial distance encoding, comprising: Calculate the radial distance from the centroid of each candidate colony region to the center of the culture dish, and encode the normalized radial distance into a position embedding vector; The location embedding vector is multiplied channel-by-channel with the feature map extracted by convolution of the local image patch of the colony candidate region to generate a location modulation feature map; An attention gating layer is set before the classification head of the colony classification network. The attention gating layer generates channel attention weights based on the position modulation feature map and weights the feature responses of different channels. The colony classification network is trained using a training set that includes samples of central flat regions and samples of edge crescent regions after geometric distortion correction and brightness normalization.

9. The system according to claim 1, characterized in that, The step of adjusting the lighting parameters of the area to be reviewed and re-acquiring images to update the recognition results includes: Analyze the identification confidence and corresponding morphological feature vector of each candidate colony in the region to be verified to determine the dominant factors of low confidence. When the dominant factor is that the grayscale contrast is lower than the preset contrast threshold, the elevation angle of the light source is increased to reduce the angle between the incident light direction and the surface normal vector of the area, thereby increasing the positive illumination intensity of the area. When the dominant factor is that the edge gradient amplitude is lower than the preset gradient threshold, switch to side lighting mode to enhance the contrast between light and dark at the edge of the colony; Based on the adjusted lighting parameters, images of the area to be reviewed are reacquired, and geometric distortion correction, brightness normalization, and colony classification are re-executed. The recognition confidence is updated until the confidence exceeds the preset confidence threshold or the preset maximum number of reviews is reached.

10. The system according to claim 1, characterized in that, The output colony locations, categories, and total counts also include: Detect cross-regional colony candidate regions located on the boundary line between the central flat region and the edge meniscus region; For the cross-regional colony candidate regions, morphological feature vectors are extracted for the portion located in the central flat region and the portion located in the edge crescent region, respectively. Calculate the feature consistency metric between the two morphological feature vectors. When the feature consistency metric is greater than a preset consistency threshold, the two parts are merged into a single colony; when the feature consistency metric is less than or equal to the preset consistency threshold, they are determined to be two independent colonies. The merged or split colonies are included in the final count result.

Images