Urine sediment image augmentation method and system based on generative adversarial network

Abstract

The application discloses a urine sediment image augmentation method and system based on a generative adversarial network, relates to the technical field of artificial intelligence image processing, and comprises the following steps: acquiring device axial constraint data of a target object, a multi-focus urine sediment image and an axial position corresponding to the multi-focus urine sediment image; extracting a position reference of the multi-focus urine sediment image and establishing a common coordinate system according to the position reference, and then mapping the multi-focus urine sediment image to the common coordinate system to generate a target basic region and a neighboring basic region. According to the application, a candidate focus interval is determined according to the change of the target region with the axial position, and each frame of defocus range is formed, so that the transmission of defocus judgment caused by the deviation of a single reference focus can be avoided; and then, the device two-dimensional influence boundary is called and merged according to the defocus range, so that the spatial exclusion range can be adaptively changed with actual out-of-focus expansion.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence image processing technology, specifically to a method and system for augmenting urine sediment images based on generative adversarial networks. Background Technology

[0002] Urine sediment microscopic image analysis is an important technical step in the detection of formed elements in urine. With the development of digital microscopic imaging, image processing and artificial intelligence technologies, using computer vision models to detect, classify or count particle targets in urine sediment microscopic images has become an important way to achieve intelligent recognition of urine sediment images.

[0003] In existing technologies, multi-focal images of urine sediment corresponding to the same field of view or the same particle can be acquired by changing the imaging position along the optical axis of the microscope. Training data can then be expanded using real images at different focal positions. This method directly preserves the defocus features actually formed by the microscope, offering the advantage of high data fidelity. To further expand the training data, image generation techniques such as generative adversarial networks can be used to learn the relationship between different image states based on existing urine sediment images and generate new augmented images. This increases the number of training samples without requiring actual acquisition for every target focal position. However, during the construction of multi-focal training data for intelligent recognition of urine sediment microscopic images, the out-of-focus response of the same particle expands beyond the main outline as the axial imaging position changes, and the degree of expansion is not fixed. The out-of-focus response of adjacent particles can also... Simultaneously entering the area surrounding the target particle, if a fixed-size background ring, fixed outward expansion distance, or single-frame segmentation boundary is used to extract the local background, it is easy to mistakenly take the out-of-focus response of the target particle itself or neighboring particles as the background. This leads to incorrect estimation of the brightness relationship, contour changes, and structural changes between the particle and the background. For example, when a particle is close to being in focus, its outer area can be used as the background, but when it is in a large out-of-focus state, its halo has extended to that outer area. If the original background area is still used, the resulting background brightness will change synchronously with the particle's own response. At the same time, the true reference focal position of the particle usually depends on the brightness and structural changes across the focal position, causing the background selection and focal position judgment to affect each other, which can easily cause the reference focal position to shift. Furthermore, it makes the out-of-focus attributes of the training samples inconsistent with the real microscopic imaging process, affecting the model's training adaptability to real out-of-focus images. Summary of the Invention

[0004] The purpose of this invention is to provide a method and system for augmenting urine sediment images based on generative adversarial networks, so as to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the technical solution of the present invention is as follows:

[0006] In a first aspect, the present invention discloses a urine sediment image augmentation method based on generative adversarial networks, applied to multifocal training data augmentation for intelligent recognition of urine sediment microscopic images, comprising the following steps:

[0007] Acquire the axial constraint data of the target object, multifocal images of urine sediment, and their corresponding axial positions;

[0008] The location reference of the multifocal urine sediment image is extracted and a common coordinate system is established accordingly. Then, the multifocal urine sediment image is mapped to the common coordinate system to generate the target base region and the adjacent base region.

[0009] Extract the axial span features of the target base region at each axial position and delineate the focal position candidate neighborhood accordingly. Then calculate the difference between the axial position of each frame in the multi-focal urine sediment image and the axial position of the focal position candidate neighborhood to generate the possible defocus range.

[0010] Query the set of two-dimensional influence boundaries that cover the possible defocus range in the axial constraint data of the device, and perform union processing on each two-dimensional influence boundary in the common coordinate system to generate a focal position space mask;

[0011] Based on the focal space mask, the affected pixels are mapped from the target base region and the adjacent base region to the periphery, and removed from the corresponding frame. Then, the remaining pixels are divided into a set of background sub-regions.

[0012] By calculating and comparing the brightness data of each background sub-region, the consistency of local illumination is verified, and then the verified background sub-regions are combined into the local final net background of the corresponding frame.

[0013] The local final net background of each frame is input into a pre-constructed generative adversarial network to generate an augmented image set.

[0014] In a second aspect, the present invention discloses a urine sediment image augmentation system based on generative adversarial networks, comprising:

[0015] The data acquisition module is used to acquire the axial constraint data of the target object, multifocal images of urine sediment and their corresponding axial positions;

[0016] The common coordinate mapping module is used to extract the position reference of the multifocal urine sediment image and establish a common coordinate system accordingly, and then map the multifocal urine sediment image to the common coordinate system to generate the target base region and the adjacent base region.

[0017] The focal position range estimation module is used to extract the axial span features of the target base region at each axial position and delineate the focal position candidate neighborhood accordingly. Then, it calculates the difference between the axial position of each frame in the multi-focal urine sediment image and the axial position of the focal position candidate neighborhood to generate the possible defocus range.

[0018] The spatial boundary fusion module is used to query the set of two-dimensional influence boundaries that cover the possible defocus range in the axial constraint data of the device, and perform union processing on each two-dimensional influence boundary in the common coordinate system to generate a focal position spatial mask.

[0019] The affected pixel removal module is used to map the focal space mask from the target base region and the adjacent base region to the periphery, mark the affected pixels and remove them from the corresponding frame, and then divide the remaining pixels into a set of background sub-regions.

[0020] The local background verification module is used to verify the consistency of local illumination by calculating and comparing the brightness data of each background sub-region, and then combine the verified background sub-regions into the local final net background of the corresponding frame.

[0021] The adversarial augmentation generation module is used to input the local final net background of each frame into a pre-constructed generative adversarial network to generate an augmented image set.

[0022] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0023] 1. This scheme determines the candidate focal position interval and forms the defocus range of each frame based on the change of the target area with the axial position, which can avoid the transmission of defocus judgment by a single reference focal position deviation; then, it calls and merges the two-dimensional influence boundary of the device according to the defocus range, so that the spatial exclusion range adapts to the actual out-of-focus expansion; accordingly, it removes the affected pixels of the target and adjacent particles, which can reduce the pollution of the background by the particle out-of-focus response; finally, it compares the brightness consistency of the remaining background partitions and forms a net background, which can suppress local illumination differences, improve the stability of background reference, and provide a reliable data foundation for subsequent focal position supervision and image augmentation.

[0024] 2. This scheme compares the brightness relationship between the target area and the temporary net background while preserving the direction of light and dark, which can avoid the absolute brightness evaluation from masking the change in focus polarity and improve the directionality of the focus position change representation; extracting the texture co-occurrence relationship of the target area to form structural repeatability can reflect the stable changes in the internal structure of the particles as they defocus; jointly representing the polarity state and structural repeatability according to the axial position can reduce misjudgments caused by a single sharpness index; determining the comparison judgment benchmark based on the actual brightness fluctuation of the temporary net background can suppress pseudo-polarity changes caused by background fluctuations; screening continuous frames that simultaneously meet the conditions of polarity reversal and structural stability along the axial direction, and limiting the candidate focus position neighborhood by their first and last positions can exclude isolated abnormal frames, so that the candidate focus position range has continuous and real focus response support. Attached Figure Description

[0025] The disclosure of this invention is illustrated with reference to the accompanying drawings. It should be understood that the drawings are for illustrative purposes only and are not intended to limit the scope of protection of this invention. In the drawings, the same reference numerals are used to refer to the same parts. Wherein:

[0026] Figure 1 A flowchart illustrating the steps of the urine sediment image augmentation method based on generative adversarial networks provided by this invention;

[0027] Figure 2 A schematic diagram of the process for generating the target base region provided by the present invention;

[0028] Figure 3 This is a schematic diagram of the process for defining the candidate neighborhood of the focal position provided by the present invention;

[0029] Figure 4 This is a schematic diagram of the process for generating augmented image sets provided by the present invention;

[0030] Figure 5 This is a schematic diagram illustrating the module functions of the urine sediment image augmentation system based on generative adversarial networks provided by the present invention. Detailed Implementation

[0031] It is readily understood that, based on the technical solution of this invention, those skilled in the art can propose various interchangeable structural methods and implementations without altering the essential spirit of the invention. Therefore, the following detailed embodiments and accompanying drawings are merely illustrative examples of the technical solution of this invention and should not be considered as the entirety of the invention or as limitations or restrictions on the technical solution of this invention.

[0032] In the process of constructing multi-focal training data for intelligent recognition of urine sediment microscopic images, the out-of-focus response of the same particle will extend beyond the main outline as the axial imaging position changes, and the degree of extension changes with the focal position. The out-of-focus response of neighboring particles may also enter the area around the target particle. If a fixed-size background ring, a fixed outward extension distance, or a single-frame segmentation boundary is used to extract the local background, the selected background area cannot correspond to the actual out-of-focus influence range of the particle at the current axial position. It is easy to use the out-of-focus response of the target particle itself or neighboring particles as background in subsequent processing. At the same time, the particle reference focal position is determined by the brightness relationship, contour change, and structural change across focal positions. When the background used to determine these changes has been affected by the particle's out-of-focus response, the background selection and the reference focal position determination will affect each other, causing deviations in the brightness relationship, contour change, and structural change between the particle and the background at different focal positions.

[0033] When the same urine sediment particle is nearly clear, a portion of its outer contour can appear as local background. When the microscope changes its imaging position along the optical axis and the particle is significantly out of focus, the halo formed by the particle's out-of-focus response will extend outward from the original main contour. If the background is still extracted using the fixed background area used when it is nearly clear, some pixels in this area actually contain the particle's own out-of-focus response. If there are neighboring particles in this area, the out-of-focus response of the neighboring particles may further enter the same background area. As a result, the background brightness obtained will change with the particle's out-of-focus response, causing the background reference originally used to describe the particle's cross-focal brightness relationship, contour state, and structural state to shift, and affecting the determination of the particle's reference focal position.

[0034] The aforementioned background reference bias continues to propagate along the multi-focal training data processing chain. After the background, contaminated by the out-of-focus response of the particles themselves or neighboring particles, participates in the judgment of brightness and dark relationships and structural changes, the state of particles at different axial positions may not correspond to their actual microscopic imaging state, causing a shift in the reference focal position. After the reference focal position shifts, the defocus attributes corresponding to the training samples will also be biased. When using such multi-focal data to expand the training data, the defocus state expressed by the generated augmented image may be inconsistent with the real microscopic imaging process. Consequently, samples with distorted focal position attributes are mixed into the training data used for urine sediment particle detection, classification, or counting, affecting the model's training adaptability to real defocused urine sediment images.

[0035] After introducing the basic concept of the present invention, the embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0036] Example 1:

[0037] This embodiment provides a urine sediment image augmentation method based on generative adversarial networks, which is applied to multi-focal position training data augmentation under intelligent recognition of urine sediment microscopic images. The method can be executed by a urine sediment image augmentation platform. The urine sediment image augmentation platform includes an image acquisition unit communicatively connected to a digital microscopic imaging device, a stage focusing unit carrying an axial drive mechanism and an axial position reading device, an image processing and model calculation unit deployed in a local workstation or server, and a data storage unit for storing calibration records and augmented samples. For ease of understanding, the following description takes the digital microscopic imaging device supporting an automatic urine formed component analyzer as the operation scenario, which does not constitute a limitation on the application scenario. The method is also applicable to manual microscopic examination digital platforms, flow imaging urine sediment analyzers, body fluid cell morphology teaching image libraries, and other microscopic imaging scenarios that need to change the imaging position along the optical axis to collect images of the same field of view.

[0038] The target objects processed in this embodiment are particle targets in the urine sediment microscopic image, including urine formed components such as red blood cells, white blood cells, epithelial cells, casts, crystals, bacteria, and mucus threads. Before the urine sediment image augmentation platform performs augmentation, it first acquires three types of initial data, namely device axial constraint data, multi-focal position urine sediment images, and their corresponding axial positions.

[0039] The multi-focal position urine sediment images refer to images of different focal position states continuously collected for the same field of view or the same particle target by changing the imaging position along the optical axis direction through the digital microscopic imaging device, which are obtained by continuously collecting the same field of view after the digital microscopic imaging device changes the imaging position stepwise along the optical axis direction. During the collection process, the objective lens magnification, light source intensity, condenser aperture, exposure time and gain settings remain unchanged, and only the stage focusing unit drives the stage or the objective lens to move along the optical axis. The axial position refers to the imaging position in the optical axis direction of the microscopic device corresponding to each frame of the multi-focal position urine sediment image, which is synchronously read by the axial position reading device at the exposure moment of each frame and bound to the frame of image, with the unit of micrometer.

[0040] The device axial constraint data is a calibration record describing the outward expansion range of the out-of-focus response of this digital microscopy imaging device at a given defocus level. It consists of a set of mapping relationships indexed by the defocus level and using the two-dimensional influence boundary as the value, and is bound to the device's hardware identifier. The two-dimensional influence boundary refers to the closed boundary formed by connecting the positions where, at a given defocus level, the out-of-focus response of the calibration object expands outward relative to the object's main contour within the imaging plane and ultimately falls back into the normal fluctuation range of the background. The device axial constraint data is acquired once during device installation or routine calibration. The acquisition method is as follows: under the same optical configuration, the axial position of the calibration object is gradually changed. For each effective axial offset, a two-dimensional response image of the calibration object and a background area image are acquired. Multiple spatial directions are set out from the center of the calibration object outwards. The grayscale fluctuations from the object's interior to its exterior are compared sequentially in each spatial direction. The position where the outer response falls back into the normal fluctuation range of the background area image is recorded as the out-of-focus influence termination position in that spatial direction. Then, the two-dimensional image coordinates of each out-of-focus influence termination position are connected to form the two-dimensional influence boundary corresponding to that axial offset. It should be noted that the two-dimensional influence boundary is not stored in absolute image coordinates, but rather the outward extension distance of the out-of-focus influence termination position relative to the main outline of the calibrated object is recorded sequentially in spatial direction. The reason for using relative outward extension distance instead of absolute coordinates is that the shape and size of the actual particles are not consistent with those of the calibrated object. If stored in absolute coordinates, it would be difficult to transfer to the actual particles. However, when stored in relative outward extension distance, it is only necessary to extend the distance along the corresponding spatial direction from the main outline of the actual particle to obtain the out-of-focus response extension range of the particle at that defocus level, regardless of the shape of the particle itself.

[0041] It should also be noted that the maximum defocus span refers to the difference between the maximum and minimum values ​​of the axial position of each frame in the multifocal urine sediment image sequence. Its physical meaning is the upper bound of the degree of defocus that any frame may have relative to the true reference focal position. The maximum two-dimensional influence boundary refers to the two-dimensional influence boundary found in the equipment axial constraint data with the maximum defocus span as the index. Its outward expansion distance in each spatial direction constitutes a spatial upper bound that is independent of the value of the true reference focal position.

[0042] The calibration step size, number of spatial directions, and interpolation method for the axial constraint data of the equipment are determined according to the following criteria: The calibration step size is the interval between two adjacent effective axial offsets. Since an excessively large step size can cause abrupt changes in the two-dimensional influence boundary between adjacent calibration points, and an excessively small step size can make calibration unacceptably time-consuming, the calibration step size is set to a range of 0.2 micrometers to 1.0 micrometers, with a typical value of 0.5 micrometers. Because the out-of-focus response is not strictly isotropic in the imaging plane, a single-direction measurement is insufficient to characterize the closed boundary. Therefore, the number of spatial directions is set to a range of 8 to 36, with a typical value of 16, and each spatial direction is evenly distributed along the center of the calibration object. For the degree of defocus between two adjacent effective axial offsets, linear interpolation is performed on the two out-of-focus influence termination positions in the same spatial direction according to the ratio of the defocus degree to the actual distance between the two adjacent effective axial offsets. After obtaining the intermediate boundary position, it is added to the axial constraint data of the equipment, thus allowing the continuously changing two-dimensional influence boundary to be queried under any degree of defocus. Once calibrated, the axial constraint data of the equipment is fixed along with the equipment hardware identifier and is only updated when the objective lens is replaced, the light source module is replaced, or when routine calibration is performed and deviations are found.

[0043] like Figure 1 As shown, the overall process framework of this embodiment consists of seven interconnected processes. First, it acquires the equipment axial constraint data, multi-focal urine sediment images, and their corresponding axial positions. Second, it extracts the positional references from the multi-focal urine sediment images and establishes a common coordinate system accordingly, mapping the multi-focal urine sediment images to the common coordinate system to generate the target base region and adjacent base regions. Third, it extracts the axial span features of the target base region at each axial position and delineates the focal position candidate neighborhood accordingly, calculating the difference between the axial position corresponding to each frame in the multi-focal urine sediment image and the axial position of the focal position candidate neighborhood to generate the possible defocus range. Finally, it queries the equipment axial constraint data to find two... The system generates a focal space mask by performing a union operation on each 2D influence boundary set in a common coordinate system. Then, based on the focal space mask, it maps from the target base region and neighboring base regions outwards, identifies the affected pixels, removes them from the corresponding frames, and divides the remaining pixels into a set of background sub-regions. Next, it verifies the local illumination consistency by calculating and comparing the brightness data of each background sub-region, and combines the verified background sub-regions into the local final net background of the corresponding frame. Finally, it inputs the local final net background of each frame into a pre-constructed generative adversarial network to generate an augmented image set.

[0044] In this context, the common coordinate system refers to the spatial reference coordinate system after uniformly mapping images from different focal positions. The target base region refers to the target subject region formed by the aggregation of stable spatial positions of the same target particle in multiple focal position images. The neighboring base region refers to the surrounding particle region located outside the target base region and potentially affected by the out-of-focus response of the target particle or neighboring particles. The axial span features of the target base region at each axial position are extracted, and based on this, candidate focal position neighborhoods are delineated. Then, the difference between the axial position of each frame in the multi-focal position urine sediment image and the axial position of the candidate focal position neighborhood is calculated to generate the possible defocus range. The axial span feature refers to the set of data features used to describe the changes in the imaging state of the target base region as the axial position changes. The candidate focal position neighborhood refers to the spatial range where the potentially clear focal position is located, determined based on the cross-axial variation law of the target particle. The possible defocus range refers to the deviation range of the corresponding axial position of each image frame relative to the candidate focal position neighborhood. The focal position spatial mask refers to the maximum spatial coverage range that the target particle may produce an out-of-focus response effect under all possible defocus states when the current reference focal position is not uniquely determined. The background sub-region set refers to the partitioned candidate background data set that has eliminated the out-of-focus effects of grains but has not yet completed the background reliability screening. The final local net background refers to the set of several local regions formed by dividing the remaining background candidate pixels according to their spatial location after removing affected pixels.

[0045] It should be noted that the focal position candidate neighborhood is not a fixed axial position value, but a spatial range defined by the axial positions of the beginning and end of a continuous image frame sequence. Correspondingly, the possible defocus range of each frame is not a fixed defocus value, but an interval defined by the difference in axial position between the frame's axial position and the near and far ends of the focal position candidate neighborhood. After preserving the interval form, each two-dimensional influence boundary in the set of two-dimensional influence boundaries covering the possible defocus range corresponds to a defocus hypothesis within that interval. The focal position spatial mask obtained by performing a union process on all two-dimensional influence boundaries is the spatial envelope covering all defocus hypotheses within the interval.

[0046] It should also be noted that the axial span feature used to define the candidate neighborhood of the focal position requires a brightness reference that does not contain particle out-of-focus response. In this embodiment, the brightness reference is taken from the pixel statistics outside the envelope obtained by extending outward from the range defined by the maximum two-dimensional influence boundary. The determination of this envelope depends only on the axial positions of the beginning and end of the sequence and does not depend on any focal position assumption. Therefore, using the pixel statistics outside of this envelope as the brightness reference is a necessary implementation condition to ensure that the definition of the candidate neighborhood of the focal position does not introduce circular dependencies. The specific construction method will be detailed later.

[0047] Based on the focal space mask, the affected pixels are mapped from the target base region and the adjacent base region to the periphery, and then removed from the corresponding frame. The remaining pixels are then divided into a set of background sub-regions.

[0048] It should be noted that the outward mapping of the focal space mask starts from both the target base region and the adjacent base region. The reason for performing the same mapping on the adjacent base region is that the adjacent particles and the target particles are under the same field of view, the same objective lens, and the same light source configuration. Their out-of-focus response is also governed by the optical characteristics of the same digital microscopy imaging device. Therefore, the two-dimensional influence boundary set covering the possible defocus range in the device axial constraint data also applies to the adjacent base region. Since the focal space mask is taken from the union of this set, its range is the maximum spatial envelope within the possible defocus range. Even if the actual axial position of the adjacent particles is not the same as that of the target particles, the extension range of their out-of-focus response in the imaging plane is still covered by this union. Therefore, using the same focal space mask for the adjacent base region is a conservative approach of taking the maximum envelope and will not result in any omissions.

[0049] By calculating and comparing the brightness data of each background sub-region, the consistency of local illumination is verified, and then the verified background sub-regions are combined into the local final net background of the corresponding frame.

[0050] It should be noted that the remaining pixels do not directly constitute the final local net background. Instead, they are first divided into a set of background sub-regions, and then optimally combined after local illumination consistency verification. This verification is added because removing affected pixels only eliminates interference from out-of-focus response, and cannot eliminate brightness deviations caused by uneven illumination, condenser lens eccentricity, differences in cover glass thickness, and suspended impurities in the liquid phase. If the average brightness of the remaining pixels is calculated as a whole, the above deviations will be averaged into the background statistics, causing a systematic shift in the subsequent brightness determination based on background brightness.

[0051] This feature addresses the problem that the obtained background brightness changes synchronously with the particle's own response, leading to an incorrect estimation of the brightness relationship between the particle and the background. This is because it changes the background composition from a continuous annular region to a set of sub-regions that are selected and retained after consistency testing. This means that the final net background in some areas does not have to be continuous in space. Thus, even when some areas are contaminated by out-of-focus response or lighting deviation, the background sub-regions in other areas can still provide a reliable background brightness reference, terminating the continued acceptance of the contaminated areas.

[0052] Through the above technical solution, this embodiment decouples the two originally interdependent processes of background extraction and reference focus determination by retrieving the device's optical calibration envelope in reverse with the possible defocus range of intervals and simultaneously constraining the spatial influence range of the target particle and its neighboring particles with the union of the envelope. Unlike the conventional processing method of extracting local background with a fixed-size background ring or single-frame segmentation boundary, this embodiment determines the spatial assignment of the background at the level of common coordinate system and device physical calibration, rather than at the level of single-frame grayscale, so that the determination result of background pixels does not drift with the defocus state of the particles themselves. The resulting local final net background is used in subsequent processing as a brightness reference for extracting the cross-focus contrast polarity and as the input channel of the generative adversarial network.

[0053] Low-pass filtering is performed on each adjacent frame in the multifocal urine sediment image to generate position references; the position coordinate difference between each position reference in adjacent frames is calculated and a residual objective function is constructed accordingly, and then the lateral translation distance that minimizes the residual objective function is solved.

[0054] It should be noted that positional reference refers to the spatial structural information extracted from multifocal urine sediment images that does not significantly change with the defocus state, such as the illumination gradient distribution within the field of view, the edge of the coverslip, large-scale textures under the microscope, and the inherent vignetting morphology of the imaging plane. Lateral translation distance refers to the two-dimensional translation correction amount between adjacent images obtained after minimizing the residual objective function.

[0055] The reason for choosing low-frequency field-of-view structure rather than grain outline as the positional reference is that the sharpness of grain outline varies drastically with the defocus state. The outline size of the same grain can differ by several times between a sharp frame and a deeply defocused frame. If the outline is used as the registration basis, the registration result will be contaminated by the defocus state itself. In contrast, the low-frequency field-of-view structure is determined by the illumination optical path and the geometric relationship of optical elements. Its position in the imaging plane remains basically unchanged during small-range movements along the optical axis, and it can serve as a positional reference across focal positions. The low-pass filter is implemented using a Gaussian filter. The standard deviation of the filter kernel is selected from one to three percent of the diagonal length of the field of view, typically two percent, so that only components with spatial frequencies lower than the characteristic scale of the grain are retained after filtering.

[0056] For example, the formula for calculating the residual objective function is as follows:

[0057] ;

[0058] in, Represent the residual objective function; Indicates the horizontal translation distance, in pixels; Indicates the first The first frame The position coordinates of each reference location, in pixels; Indicates the first The first frame The position coordinates of each reference location, in pixels; Indicates the number of location references; Indicates the position reference number; Indicates the frame number; This represents Euclidean norm operations.

[0059] When determining the lateral translation distance that minimizes the residual objective function, a global least squares approach is used. This means simultaneously optimizing the position coordinate differences of all position references, rather than directly using the position coordinate difference of any single position reference as the translation amount. It's important to note that the number of position references is limited; too few values ​​can lead to the solution being affected by local deformation of individual structures, while too many values ​​can introduce interference between low-frequency structures. Therefore, the number of references is set to a range of 4 to 16, with a typical value of 8. The obtained lateral translation distance is the position correction amount between adjacent frames. Translating each adjacent frame according to this position correction amount completes the mapping to the common coordinate system. The translated image frame is denoted as the aligned frame.

[0060] Extract the particle center positions and their relative particle arrangement topology from each alignment frame; calculate the spatial distance difference between particle center positions between alignment frames and compare the structural consistency of each particle relative arrangement topology.

[0061] It should be noted that the extraction of particle center positions targets the low-frequency main body region that persists across the focal length of the particle, rather than the complete response region of the particle in the current frame. The particle center position refers to the two-dimensional coordinate data representing the spatial position of a particle's main body center in the common coordinate system of the aligned frames. The low-frequency main body region refers to the gray-level cluster area that remains connected after low-pass filtering of the aligned frame and exists continuously in several adjacent frames, corresponding to the solid part of the particle. Ring-shaped or halo-shaped responses that are independent of the low-frequency main body region and only appear in out-of-focus frames, expanding and contracting with the degree of defocusing, are considered out-of-focus responses and are excluded when extracting the particle center position. The relative arrangement topology of particles refers to the geometric structure formed by the interconnected lines of all particle center positions within the same aligned frame. In this embodiment, the ratio of the relative azimuth angle and relative distance between each particle center position and its nearest several particle center positions is used to characterize this geometric structure, thus ensuring that this characterization remains unchanged under overall translation and scaling.

[0062] For example, the formula for calculating spatial distance difference is:

[0063] ;

[0064] in, Indicates the first The first frame Spatial distance difference between the center positions of individual particles, unit: pixels; Indicates the first The first frame The center position of each particle, in pixels; Indicates the first The first frame The center position of each particle, in pixels.

[0065] Extract the set of particle locations distributed around the target base region in a common coordinate system to generate the neighboring base region.

[0066] To further clarify, the extraction range of the adjacent base region starts from the boundary of the target base region and extends outward to the distance defined by the maximum two-dimensional influence boundary. All particles whose center position falls within this extended range are included in the adjacent base region.

[0067] Through the above technical solution, this embodiment uses a low-frequency field-of-view structure that is insensitive to defocus as the cross-focal-position registration reference and establishes the cross-frame particle identity by dual matching of the particle center position and the relative arrangement topology of the particles. This transforms the spatial assignment of the same particle in different defocus states from a single-frame segmentation problem into a cross-frame correspondence problem in a common coordinate system. Unlike the conventional processing method of independently performing segmentation in each frame and associating the results of adjacent frames according to the overlap rate, this embodiment first removes the high-frequency components dominated by the defocus state when selecting the registration reference, so that the registration accuracy does not fluctuate with the sharpness of the particles. The target base region and the adjacent base region obtained in this way are used as two starting points for mapping the focal-position spatial mask to the outside in subsequent processing.

[0068] like Figure 2 As shown, the set of particle positions is not directly aggregated from the particle center positions that establish the corresponding mapping. Instead, it is first aggregated into an initial set of particle positions, and then generated after trajectory verification to remove abnormal frames.

[0069] The number of corresponding mappings of the initial particle position set between adjacent frames is counted, and the corresponding number of mappings is recorded as the trajectory topology state; when the trajectory topology state exhibits a preset abnormal state, a spatial abnormal event is determined to have occurred.

[0070] It should be noted that the trajectory topology state refers to the particle identity continuity state formed by the number of corresponding mappings between adjacent frames. A corresponding mapping refers to the one-to-one correspondence established between the center positions of two particles with the smallest spatial distance difference and structural consistency (matching) between two adjacent aligned frames; the number of corresponding mappings is the number of corresponding mappings a given particle center position participates in between adjacent frames. Ideally, the same particle should have only one corresponding mapping between adjacent frames, meaning the number of corresponding mappings is always one. However, when the liquid phase flow in the urine sample causes lateral drift of the particles, or when adjacent particles occlude each other after defocusing, a situation may arise where one particle center position simultaneously corresponds to multiple particle center positions in the next frame, or multiple particle center positions simultaneously correspond to the same particle center position in the next frame. There may also be situations where particles move out of the common effective field of view without any corresponding mappings.

[0071] It should be further explained that the preset abnormal state is a criterion used to determine whether the trajectory topology state deviates from the normal cross-frame correspondence, and is used to trigger the determination of spatial abnormal events. Its contents include the following: First, the number of corresponding mappings is greater than one, corresponding to one-to-many splitting or many-to-one merging; Second, the number of corresponding mappings is equal to zero, corresponding to being moved out of the common effective field of view or being completely occluded. It should be noted that when the spatial distance difference between the center positions of two particles exceeds the particle drift tolerance, even if the structural consistency of the two is matched, no corresponding mapping will be established between them, and the number of corresponding mappings at the center position of the particle will be reduced to zero, thus falling into the second type of situation mentioned above. After this processing, the trajectory topology state is represented by a single value of the number of corresponding mappings in all cases, without introducing a determination dimension other than the number of corresponding mappings. Since the corresponding mapping quantity is a count quantity, its normal value is uniquely determined to be one, and there is no need to set a separate value range. Since the particle drift tolerance is used to distinguish between normal liquid phase micro-motion and abnormal particle track switching, if the value is too small, normal micro-motion will be misjudged as abnormal, and if the value is too large, true incorrect pairing will be missed. Therefore, the value range of the particle drift tolerance is set to 10% to 30% of the equivalent diameter of the target base area, with a typical value of 20% of the equivalent diameter of the target base area.

[0072] Remove adjacent frames from the initial set of particle locations where spatial anomalies occur, and then aggregate the particle center locations in the remaining frames after removing adjacent frames into a new set of particle locations.

[0073] It should be noted that the object to be truncated is the adjacent frame where the spatial anomaly event occurred, that is, the frame located at the end of the axial sequence in the image frames at both ends of the corresponding mapping; after truncation, the image frames before and after this frame still retain their respective particle center positions, but a corresponding mapping across the truncated frame is no longer established between the two segments.

[0074] Extract the extreme values ​​of the outer coordinates of the particle location set in each direction within the common coordinate system to generate the enclosing boundary, and generate the target base region based on the spatial region delineated within the common coordinate system according to the enclosing boundary.

[0075] To further explain, the extraction of the extreme values ​​of the outer coordinates is performed along the horizontal and vertical directions of the common coordinate system. The maximum and minimum values ​​of the pixel coordinates of the low-frequency main body region corresponding to the center positions of all particles in the particle position set are taken, and the rectangle formed by the four extreme values ​​is the bounding boundary.

[0076] Through the above technical solution, this embodiment uses the corresponding mapping quantity as a measure of the integrity of cross-frame particle identity and performs frame-by-frame truncation on the initial particle position set accordingly. This isolates segments in the particle trajectory that are distorted due to liquid phase flow and mutual occlusion before entering the region delineation. Unlike the conventional processing method of directly taking the intersection or union of the segmentation results of all frames to determine the particle range, this embodiment performs anomaly removal at the trajectory level rather than the pixel level, so that a small number of abnormal frames do not enlarge or compress the overall boundary of the target base region. The target base region obtained in this way is used as a fixed statistical range for calculating the average pixel brightness and gray-level co-occurrence matrix in subsequent processing.

[0077] Before defining the candidate neighborhood of the focal position, a temporary net background needs to be constructed first. The reason why it is necessary to construct a temporary net background in advance is that the brightness polarity state in the axial span feature requires a background brightness reference to be extracted, and this reference does not exist before the local final net background is obtained; if the average brightness of the whole frame is used directly as a substitute, the average brightness includes the out-of-focus response of the grain itself and the neighboring grains, and its value changes with the out-of-focus state. The brightness polarity state extracted with this as a reference will lose its physical meaning.

[0078] The maximum and minimum values ​​in the axial positions of each frame are extracted and their differences are calculated to generate the maximum defocus span. The maximum two-dimensional influence boundary corresponding to the maximum defocus span is queried from the device axial constraint data. Under the common coordinate system, the range defined by the maximum two-dimensional influence boundary is extended outward along the target base area to generate a focal position-independent conservative spatial envelope. The focal position-independent conservative spatial envelope refers to the focal position-independent conservative spatial envelope.

[0079] Since the axial position difference between the true reference focal position and the actual reference focal position in any frame will not exceed the difference between the axial positions at the beginning and end of the sequence, the maximum two-dimensional influence boundary obtained by querying the maximum defocus span must have an outward expansion distance in each spatial direction that is not less than the true out-of-focus response expansion distance of any frame.

[0080] When extending outward along the target base region, each point on the boundary of the target base region is taken as the starting point, and the distance of the maximum two-dimensional influence boundary in each spatial direction within the common coordinate system is extended. The closed region formed by connecting the endpoints of each extension is the focal position independent conservative spatial envelope.

[0081] Pixels located within the focal-independent conservative spatial envelope are removed from multifocal urine sediment images. The remaining pixels that are not removed are then extracted as a temporary background candidate pixel set. The temporary background candidate pixel set refers to the set of all background pixels remaining after removing pixels within the focal-independent conservative spatial envelope from the multifocal urine sediment images.

[0082] Calculate the mean and standard deviation of the brightness of the temporary background candidate pixel set, and construct a brightness fluctuation range defined by the mean and standard deviation. Filter the pixels in the temporary background candidate pixel set whose brightness values ​​fall within the brightness fluctuation range, and combine them to form a temporary net background. The temporary net background refers to the set of background pixels obtained after filtering the temporary background candidate pixel set through the brightness fluctuation range.

[0083] For example, the expression for calculating the brightness fluctuation range is:

[0084] ;

[0085] in, This represents the average brightness level, expressed in grayscale levels. Indicates the standard deviation of brightness, in gray levels; This represents the coefficient for the brightness fluctuation range.

[0086] It should be noted that if the brightness fluctuation range coefficient is too small, normal sensor noise and minor illumination fluctuations will be eliminated, resulting in an insufficient number of temporary net background pixels. If the value is too large, it will be difficult to exclude isolated abnormal pixels such as liquid impurities and bubble edges. Therefore, the range of the brightness fluctuation range coefficient is set to 1.5 to 3.0, with a typical value of 2.0. This typical value corresponds to retaining approximately 95% of normal background pixels under an approximately normal distribution. The output of the brightness fluctuation range is used to filter the candidate pixel set for temporary background, and the filtering result is the temporary net background. The average brightness value itself is directly used in the subsequent extraction of brightness polarity state as a comparison benchmark for the average brightness value of pixels in the target base area.

[0087] It should be noted that there is a lower limit requirement for the number of pixels in the temporary net background. When the number of pixels after filtering is less than 2% of the total number of pixels in the whole frame, it means that the focal position-independent conservative spatial envelope has occupied most of the field of view. At this time, the frame does not have the conditions to provide a reliable brightness reference. The temporary net background of the frame is marked as invalid and replaced by the average brightness of the nearest valid frame on the axial sequence. The reference contrast threshold required for the subsequent frame is taken from the temporary net background of the same nearest valid frame.

[0088] Through the above technical solution, this embodiment obtains a physically reliable initial brightness reference at a stage where the reference focal position is not yet determined by directly retrieving the upper spatial boundary using the difference between the axial positions of the beginning and end of the sequence and constructing a conservative spatial envelope independent of the actual reference focal position value. Unlike conventional processing methods that use whole-frame statistics or fixed annular regions as initial background references, this embodiment uses extreme value envelopes rather than empirical distances to determine the sampling range, ensuring the reliability of the initial reference is guaranteed by the device's optical calibration rather than by empirical assumptions. The resulting temporary net background and its average brightness are used in subsequent processing as a comparison benchmark for extracting brightness polarity states and as a source of values ​​for determining the benchmark contrast threshold.

[0089] like Figure 3 As shown, the axial span feature is composed of two components: the brightness polarity state and the internal structure repetition. After the two are bound according to the axial position, they form a feature sequence arranged along the axial direction. The focal position candidate neighborhood is defined by the continuous image frame sequence that meets the joint condition in the feature sequence.

[0090] Calculate the average pixel brightness of the target base region in each frame, and subtract it from the average brightness of the temporary net background. Extract the sign of the difference result to generate the brightness polarity state. The brightness polarity state refers to the direction state in which the target base region is brighter or darker than the background region.

[0091] For example, the formula for calculating the light and dark polarity states is:

[0092] ;

[0093] in, Indicates the first The brightness polarity of a frame; Indicates the target base region in the th Average pixel brightness within a frame, in gray levels; This indicates the sign operation.

[0094] The brightness polarity state can be positive or negative. Positive one means that the target's base area in the frame is brighter than the background, and negative one means that the target's base area in the frame is darker than the background.

[0095] The gray-level co-occurrence matrix of the target base region is calculated in each frame, and the contrast value of the gray-level co-occurrence matrix is ​​extracted to generate the internal structure repetition. The internal structure repetition refers to the structural stability description data formed by the degree of texture change within the target base region.

[0096] For example, the formula for calculating the internal structure repeatability is:

[0097] ;

[0098] in, Indicates the first Frame internal structure repetition, unit: gray level; Indicates the first The gray-level co-occurrence matrix of the target base region in the frame is in gray-level pairs Normalized values ​​at; Represents the row gray levels of the gray-level co-occurrence matrix; This represents the column gray levels of the gray-level co-occurrence matrix.

[0099] It should be noted that the gray-level co-occurrence matrix is ​​averaged after statistical analysis in four directions with a single pixel step size. Before the statistical analysis, the gray levels of the target base region are compressed to sixteen levels to suppress the influence of noise on the co-occurrence count. The contrast ratio of the gray-level co-occurrence matrix is ​​itself a weighted sum of the squares of the gray level differences, with the dimension being the square of the gray level. The reference contrast threshold for subsequent comparison is taken from the brightness range difference, with the dimension being the first power of the gray level. To enable direct comparison between the two, this embodiment takes the square root of the contrast ratio as the internal structure repeatability, normalizing its dimension to the first power of the gray level. Its physical meaning is the equivalent gray level difference between adjacent pixels within the target base region. After this normalization process, the comparison between the internal structure repeatability and the reference contrast threshold has a clear physical meaning, namely, whether the structural undulations within the particles significantly exceed the brightness undulations of the background itself.

[0100] Extract the maximum and minimum values ​​of the brightness of all pixels within the temporary net background and calculate the difference to generate a baseline contrast threshold.

[0101] It should be noted that the baseline contrast threshold is directly obtained from the temporary net background of the current frame, and therefore automatically changes with the turbidity of the sample, the light source intensity and the exposure settings, without the need to readjust parameters for different devices or different samples.

[0102] Along the sequence direction of the corresponding axial position of each frame, extract a continuous image frame sequence in which the sign of the brightness polarity state is reversed and the internal structural repetition of each frame is greater than the benchmark contrast threshold; define the spatial range according to the axial position of the beginning and end of the continuous image frame sequence to generate a focal position candidate neighborhood.

[0103] It should be noted that the extraction of a continuous image frame sequence requires the simultaneous fulfillment of two conditions, neither of which can be omitted. The first condition is that the brightness polarity state undergoes a sign reversal within the sequence, meaning that there exist two adjacent frames where the brightness polarity state of the previous frame is opposite to that of the next frame. The physical fact corresponding to this condition is that when a particle passes through a reference focal point, its brightness direction relative to the background is reversed, and the reversal must occur near the reference focal point. The second condition is that the internal structure repetition of each frame contained in the sequence is greater than the baseline contrast threshold. The physical fact corresponding to this condition is that the internal structure of a particle can be distinguished when it is close to the reference focal point, while when it is far out of focus, the internal structure is diffused into a uniform halo response, and the internal structure repetition decreases to the level of background undulation.

[0104] The reason for using both conditions in combination is that either condition alone can fail: when using only the sign reversal condition, the brightness difference between particles and the background in the deep defocus region is close to zero, and their brightness polarity will randomly jump between positive and negative due to noise, generating a large number of false sign reversal positions; when using only the internal structure repeatability condition, some high-refractive-index crystalline particles can exhibit high internal structure repeatability in both defocus regions, making it difficult to uniquely determine the side where the reference focal position is located. After combining the two conditions, only brightness reversals that occur when the internal structure of the particles is still distinguishable are accepted, thus constraining the reference focal position to a continuous axial segment that has both polarity reversal and structural distinguishability.

[0105] The focal position candidate neighborhood is the spatial range defined by the axial positions of the first and last ends of the continuous image frame sequence, denoted as . ,in This indicates the proximal axial position of the focal position candidate neighborhood. This indicates the far-end axial position of the candidate focal position neighborhood, with units of micrometers.

[0106] Additionally, regarding the handling of situations where the brightness polarity state has not undergone sign reversal: When the particle target is a high-refractive-index crystal or a dense tubular shape, it may always be brighter or darker than the background throughout the entire acquisition axis, making it difficult to satisfy the sign reversal condition. If no processing is performed, the continuous image frame sequence will be an empty set, the focal position candidate neighborhood will be difficult to generate, and all subsequent processing will be interrupted. In this case, the following method is used to define the neighborhood: extract the longest continuous image frame sequence with an internal structure repeatability greater than the reference contrast threshold, and define the spatial range as the focal position candidate neighborhood by the axial position of the beginning and end of the sequence. Since the determination of the reference focal position in this case depends entirely on the internal structure repeatability, its interval width is usually larger than the interval width when there is sign reversal. Correspondingly, the range of the focal position spatial mask obtained subsequently is also larger, which is a conservative deviation and will not cause the omission of out-of-focus responses.

[0107] Calculate the difference between the axial position of each frame in the multifocal urine sediment image and the axial position of the candidate focal position neighborhood to generate the possible defocus range.

[0108] For example, the possible defocus range can be calculated as follows:

[0109] ;

[0110] in, Indicates the first Possible out-of-focus range of a frame, in micrometers; This indicates taking the smaller value; This indicates taking the larger value; Indicates the first The axial position of the frame.

[0111] It should be noted that when the first When the axial position of a frame falls within the candidate focal position neighborhood, the degree of defocus relative to the true reference focal position may be as small as zero. In this case, the lower bound of the possible defocus range is zero, and the upper bound is the larger of the difference between the axial position of the frame and the axial positions at both ends of the candidate focal position neighborhood. The output of the possible defocus range is used to query the set of two-dimensional influence boundaries covering this range in the device axial constraint data. All two-dimensional influence boundaries whose indices fall within this range are included in the set. Then, the union of the coverage areas of each two-dimensional influence boundary in the set in the common coordinate system is performed to obtain the focal position space mask of the frame. Since the two-dimensional influence boundaries monotonically expand outward with the degree of defocus, this union is actually dominated by the two-dimensional influence boundary corresponding to the upper bound of the possible defocus range. However, the description of taking the union of sets can accommodate the case where there are non-monotonic calibration records in the device axial constraint data.

[0112] Through the above technical solution, this embodiment binds the signed brightness direction with the dimensionally normalized internal structure discernibility as a joint feature arranged along the axis, and uses the simultaneous occurrence of sign reversal and structural discernibility as the segment truncation condition, transforming the determination of the reference focal position from single-frame scalar optimization to segment definition on a cross-frame sequence. Unlike the conventional processing method of determining the reference focal position by taking the peak value of the sharpness index along the axis, this embodiment processes the determination at the integrity level of the judgment object, that is, it uses a continuous image frame sequence rather than an extreme point to carry the judgment result, so that the fluctuation of the single-frame index caused by noise is insufficient to change the segment boundary. The resulting focal position candidate neighborhood is used as the near and far end reference for calculating the possible defocus range of each frame in subsequent processing.

[0113] Based on the focal space mask, the affected pixels are mapped from the target base region and the adjacent base region to the periphery, and then removed from the corresponding frame. The remaining pixels are then divided into a set of background sub-regions.

[0114] It should be noted that the outward mapping is performed as follows: Starting from each point on the boundary between the target base region and the boundary of the set of particle positions in the adjacent base regions, the focal space mask of the frame is extended along each spatial direction within the common coordinate system by the distance defined in the corresponding direction. All pixels covered by this extension, along with pixels within the starting region, are marked as affected pixels and removed from the frame. The remaining pixels after removal are divided into several background sub-regions according to the principles of spatial connectivity and scale matching. The division scale matches the actual spatial size of the target base region; that is, the equivalent diameter of a single background sub-region is 0.8 to 1.5 times the equivalent diameter of the target base region, with a typical value of 1.0 times. The reason for using a division scale comparable to the target base region is that the brightness data of the background sub-regions is subsequently used as the comparison benchmark for the average pixel brightness of the target base region. When the statistical areas of the two are comparable, the degree of influence of random noise on each is comparable, and the comparison results will not be systematically biased due to significant area differences.

[0115] By calculating and comparing the brightness data of each background sub-region, the consistency of local illumination is verified, and then the verified background sub-regions are combined into the local final net background of the corresponding frame.

[0116] It should be noted that the local illumination consistency verification process is as follows: For each background sub-region, the pixel grayscale values ​​are sorted and truncated using a sorting and truncation filter. After sorting from smallest to largest, extremely bright and extremely dark pixels at both ends are removed. The truncation ratio ranges from 1% to 5%, with a typical value of 2%, to exclude isolated abnormal pixels contributed by suspended impurities in the liquid phase, bubble edges, and sensor dead pixels. The arithmetic mean of the remaining truncated pixels is calculated as the center brightness of the background sub-region. The maximum and minimum center brightness values ​​of all background sub-regions are calculated and their difference is obtained to obtain the maximum difference in center brightness. The maximum difference in center brightness is compared with the illumination consistency tolerance. If the former is not greater than the latter, all background sub-regions are considered to have passed the local illumination consistency verification. If the former is greater than the latter, the background sub-region whose center brightness deviates the furthest from the median center brightness of all background sub-regions is removed, and the maximum difference in center brightness is recalculated for the remaining background sub-regions. This process is repeated until the maximum difference in center brightness is no greater than the illumination consistency tolerance. The finally retained background sub-regions are combined to form the final local net background of the corresponding frame.

[0117] To clarify, the illumination consistency tolerance is a threshold used to determine whether the brightness of different background sub-regions can be considered as the same illumination condition. It is used to determine whether a region passes or fails the local illumination consistency verification. If the tolerance value is too small, a large number of qualified background sub-regions will be mistakenly eliminated under normal field-of-view illumination gradients, resulting in insufficient net background area in the final local area. If the value is too large, it will be difficult to eliminate the unevenness of real illumination caused by condenser lens eccentricity and cover glass tilt. Therefore, the range of the illumination consistency tolerance is set from 5 gray levels to 20 gray levels, with a typical value of 10 gray levels. This typical value is determined by three times the standard deviation of the measured background area brightness under the current imaging conditions. That is, before formal acquisition, several frames are acquired in the blank field of view, and the standard deviation of their brightness is statistically analyzed, and then three times is taken as the tolerance. Thus, the tolerance depends on the actual noise level and does not rely on fixed empirical values.

[0118] Furthermore, the successive elimination process has a termination protection. When the number of retained background sub-regions drops to three, elimination stops. Regardless of whether the maximum difference in center brightness still exceeds the illumination consistency tolerance, the currently retained background sub-regions are combined as the local final net background, and the frame is marked as poorly lit. Frames marked as poorly lit do not participate in the selection of the target reference image. Their local final net background can still be used as one of the inputs of the multi-channel tensor, but when inputting into the network, its effective bit mask is set according to the actual range of the retained background sub-regions. The reason for setting this protection is that when the number of background sub-regions is too small, the statistical significance of their center brightness decreases. Continuing to eliminate will only degenerate the local final net background into a single-directional sampling, thus losing the significance of cross-verification by multiple sub-regions.

[0119] The mean background brightness of the final local net background is taken as the area-weighted average of the center brightness of all verified background sub-regions, denoted as . , indicating the first Average background brightness of the frame, in gray levels.

[0120] It should be noted that the remaining pixels used to divide the background sub-regions are the culling results after the affected pixels are calibrated with a focal space mask, and the focal space mask is derived from the union of the two-dimensional influence boundary sets retrieved from the possible out-of-focus range. An additional beneficial effect of this linkage is that the pixels entering the local illumination consistency verification are determined to be free of the out-of-focus response of the target particles and neighboring particles. Therefore, the center brightness difference detected during the verification process can be definitively attributed to uneven illumination or liquid phase impurities, rather than to the intrusion of the out-of-focus response. This gives the culling decision a clear physical direction and prevents the wrong culling of the real background that has become darker or brighter due to out-of-focus.

[0121] Through the above technical solution, this embodiment performs sorting and truncation and center brightness comparison on background sub-regions that match the scale of the target base region, and successively eliminates them according to the degree of deviation from the median. The composition of the final local net background is changed from a spatially continuous ring-shaped region to a combination of sub-regions selected and retained after consistency verification. Unlike the conventional processing method of calculating the average brightness of the entire ring-shaped region around the particles, this embodiment introduces a mutual verification mechanism between sub-regions at the composition level of background statistics, so that the background brightness average of the whole frame is not polluted when individual locations are affected by uneven lighting or impurities. The resulting final local net background and its average background brightness are used as a comparison benchmark for determining the polarity of the focus contrast and as the input channel of the generative adversarial network in subsequent processing.

[0122] like Figure 4 As shown, the process of inputting the local final net background of each frame into the pre-constructed generative adversarial network to generate an augmented image set consists of six interconnected steps: conditional quantity extraction, target reference image selection, multi-channel tensor construction, image prediction residual generation, candidate target focal position image fusion, and stateful sequence backtesting verification.

[0123] Calculate the average background brightness of the final net background in the local area, and calculate the difference between the average pixel brightness and the average background brightness. Extract the sign of the difference calculation result to generate the focus contrast polarity.

[0124] It should be noted that while the focus-through contrast polarity is similar in calculation form to the aforementioned brightness-darkness polarity, their comparison benchmarks and uses are different and should not be confused: the brightness-darkness polarity uses the average brightness of the temporary background candidate pixel set as the comparison benchmark, taken from far-end pixels outside the focal-position-independent conservative spatial envelope, to define the focal-position candidate neighborhood; the focus-through contrast polarity uses the average background brightness of the local final net background as the comparison benchmark, taken from near-end pixels that have been precisely eliminated by the focal-position spatial mask and verified by local illumination consistency, and is used as a conditional quantity for the generative adversarial network and as a comparison benchmark for subsequent stateful sequence backtesting. Since the comparison benchmark converges from a far-end coarse reference to a near-end precise reference, the focus-through contrast polarity is closer to the true brightness-darkness relationship between particles and their immediate background than the brightness-darkness polarity. In several frames where particles are close to the reference focal position and the brightness-darkness difference is small, the two may take different values; in this case, the focus-through contrast polarity shall prevail.

[0125] Calculate the grayscale gradient of the target base region in each frame and extract the maximum value to generate the granular structure state.

[0126] For example, the formula for calculating the particle structure state is:

[0127] ;

[0128] in, Indicates the first The granular structure state of a frame, in gray levels per pixel; Indicates the first The image of the frame; This represents the partial derivative operator along the horizontal axis of the common coordinate system; This represents the partial derivative operator along the longitudinal direction of the common coordinate system; This represents the set of pixels covered by the target's base region; Represents the horizontal coordinate of the common coordinate system, in pixels; Represents the vertical coordinate of the common coordinate system, in pixels.

[0129] It should be noted that the partial derivative operator is implemented by the third-order Sobel operator, with one set of third-order convolution kernels in both the horizontal and vertical directions performing convolution on the target base region and then taking the modulus. Since the subsequent stateful sequence back-verification requires direct numerical comparison between the generated structural state of the candidate target focal position image and the granular structure state of the acquired frame, the two must be calculated by the same operator to be comparable. Therefore, the same partial derivative operator is used for each acquired frame and each candidate target focal position image throughout the entire process, and different operators or different kernel sizes must not be mixed.

[0130] The state of particle structure reflects the steepness of grayscale transition at the particle outline. Its value decreases monotonically as the degree of defocus increases. This monotonicity is determined by the law of high-frequency response decay of optical transfer function when defocused. It is the physical basis for determining whether the generated structure state is reasonable in the subsequent state sequence back-verification.

[0131] The frame with the largest absolute value of the difference calculation result is selected as the target reference image, and the axial position of the uncollected data within the possible out-of-focus range is specified as the target focal position to be generated.

[0132] It's important to note that the target reference image is chosen based on the largest absolute difference between the average pixel brightness and the average background brightness, rather than the one with the largest particle structure state. This is because the generative adversarial network's task here is to predict how the brightness distribution of particles relative to the background changes with axial position. The frame with the largest absolute value provides the highest signal-to-noise ratio comparison between particle and background brightness, and its pixel distribution is least affected by background noise. While the frame with the largest particle structure state has the sharpest outline, the brightness difference between particles and the background in that frame might be close to zero. Using it as a reference would require subsequent image prediction residuals to handle a significant change from zero contrast to significant contrast, increasing the mapping difficulty. The target focal position to be generated is selected within the axial range covered by the possible defocus range, between two adjacent acquired axial positions. Its value must not coincide with the axial position of any acquired frame to ensure that the generated result supplements the existing training data rather than repeats it.

[0133] The local final net background of each frame, the target reference image, the target focal position to be generated, the focus contrast polarity, and the grain structure state are combined into a multi-channel tensor.

[0134] To further explain, the multi-channel tensor is constructed as follows: the target reference image is used as the first channel; the local final net background of each frame is laid in ascending order of its corresponding axial position and in accordance with its spatial position in the common coordinate system, forming a local final net background channel group. Positions within each frame that fail the local illumination consistency verification are filled with zero values, and a corresponding effective bit mask channel group is set up in the same order to indicate to the network frame by frame which spatial positions have reliable background values; the difference between the target focal position to be generated and the axial position of the target reference image is broadcast as a channel of the same size as the image; the focus contrast polarity is broadcast as a channel; and the granular structure state is broadcast as a channel after global normalization. The reason for broadcasting the scalar conditional quantity as a channel of the same size as the image, rather than injecting it as a vector in the middle layer of the network, is to ensure that the convolution operation can access the conditional quantity simultaneously at every spatial position, avoiding dilution of conditional information during the expansion of the receptive field in deep convolutions.

[0135] It should also be noted that the reason for including the local final net background of each frame, rather than only the local final net background of the frame corresponding to the target reference image, is that the illumination distribution within the same field of view changes slowly along the axial direction as the relative position of the condenser lens and the stage changes. A single frame background alone cannot reflect this trend. By including each frame, the network can obtain the variation law of illumination distribution with axial position from the differences in the local final net background channel group along the frame dimension, thus providing a background representation consistent with this law at the target focal position to be generated. When only the local final net background of the frame corresponding to the target reference image is available in the acquisition sequence, the local final net background channel group degenerates into a single frame, which is a simplified case of taking only one frame; the rest of the scheme remains unchanged.

[0136] Through the above technical solution, this embodiment establishes a signed cross-focus contrast polarity based on the accurately converged mean background brightness and selects the target reference image by comparing brightness with signal-to-noise ratio rather than contour sharpness, fixing the starting point of the generation task at the frame where the relationship between grain and background brightness is most discernible. Unlike the conventional processing method that uses a sharp frame as the generation source and unsigned sharpness as a conditional quantity, this embodiment retains the sign information of the brightness direction at the level of conditional quantity composition and binds it to the steepness of the structure, so that two completely different imaging states on both sides of the reference focal position under the same degree of defocus can be distinguished. The resulting multi-channel tensor is used as the sole input to the image prediction residual generated by the generative adversarial network in subsequent processing.

[0137] By using convolutional layers within a generative adversarial network, feature map vectors are extracted from the target baseline image and the local final net background, respectively. The focal position, focus contrast polarity, and granular structure state of the target to be generated are concatenated along the channel dimension to generate a global conditional vector.

[0138] Generative Adversarial Networks (GANs) consist of a generator and a discriminator. The generator employs an encoder-decoder structure, with two parallel convolutional branches at the encoder end. The first convolutional branch takes the target reference image as input and extracts the reference image features corresponding to the particle body, contour, and local texture through convolution and downsampling. The second convolutional branch takes the final net background and its effective bit mask of each frame as input, extracts background features frame by frame using shared-weight convolution operations, and performs mean aggregation and range aggregation along the frame axis before channel stitching to obtain background features that simultaneously represent the average background illumination state and its axial variation.

[0139] The target focal position, focus contrast polarity, and granular structure state are combined to form a global condition vector. The scaling factor and bias factor corresponding to each level of feature channel of the decoding end are obtained through fully connected mapping. The decoding end fuses the features obtained from the two coding branches and performs affine modulation on the current layer features using the scaling factor and bias factor during the upsampling process until the spatial size is restored to the same as the target reference image and the single-channel initial brightness change matrix is ​​output.

[0140] The initial brightness variation matrix is ​​multiplied pixel-by-pixel with the focal space mask corresponding to the target focal position to be generated, suppressing brightness variations outside the mask's defined range. The resulting matrix serves as the image prediction residual and is fused pixel-by-pixel with the target reference image to form a candidate target focal position image. The discriminator employs a convolutional feature extraction structure corresponding to the generator's encoding end, using both the real target focal position image and the candidate target focal position image as discriminant inputs, and simultaneously introducing the same global conditional vector to judge the authenticity of different inputs under specified focal position and particle state conditions. During training, adversarial loss and structural penalty loss used to constrain the particle's main body position, contour, and internal structure are combined to alternately update the generator and discriminator parameters.

[0141] It should be noted that the generator of the generative adversarial network adopts an encoder-decoder structure. The encoder has two parallel convolutional branches. The first branch takes the target reference image as input, and the second branch takes the local final net background channel group along with the effective bit mask channel group as input. Each branch obtains its own feature map vector after passing through several convolutional layers and downsampling layers. The second branch extracts each frame in the local final net background channel group frame by frame using a convolutional kernel with shared weights. Then, it performs aggregation operations of averaging and range along the frame dimension and concatenates the results. This allows the output channel number to remain unchanged with the number of frames while preserving the average level of the illumination distribution and its variation along the axis. The reason for using dual-path extraction instead of simply superimposing the two into a multi-channel input is that the target reference image contains high-frequency components of the main grain and out-of-focus response, while the local final net background only contains low-frequency components of illumination distribution and noise. The spatial frequency characteristics of the two types of components are significantly different. Sharing the same set of convolution kernels will cause the low-frequency background components to be submerged by the response of the high-frequency components in the shallow layer. Dual-path extraction can ensure that the illumination distribution information carried by the background is completely preserved until the fusion stage.

[0142] The initial brightness change matrix is ​​generated by performing matrix operations of multiplication and addition on the feature map vector using the global conditional vector.

[0143] For example, the formula for calculating the initial brightness change matrix is:

[0144] ;

[0145] in, This represents the initial brightness change matrix, in gray levels. This represents the stepwise upsampling and convolution mapping at the generator's decoding end; This represents the channel-wise scaling factor obtained by mapping from the global condition vector; Represents the channel-wise bias coefficient obtained by global condition vector mapping, in gray levels; The feature map vector representing the target reference image; The feature map vector representing the local final net background channel group; This represents element-wise multiplication. This represents the global condition vector.

[0146] It should be noted that after downsampling at the encoder, the spatial size of the feature map vector is smaller than that of the target reference image, while the focal space mask is defined on the same spatial size as the target reference image in a common coordinate system. Therefore, they cannot be directly multiplied pixel-by-pixel. For this reason, the matrix operations of multiplication and addition are not performed only once at the single resolution output at the encoder, but are performed sequentially at the decoder stage. That is, after each level of upsampling, the feature map vector at the current resolution is multiplied and added again using the channel-by-channel scaling coefficients and channel-by-channel bias coefficients derived from the same global condition vector, until the spatial size of the final level output is restored to the same as that of the target reference image and the number of output channels is one. This output is the initial brightness variation matrix. After this sequential execution, the initial brightness variation matrix and the spatial size of the focal space mask are strictly consistent, and the correspondence of pixel-by-pixel multiplication is clear.

[0147] It should be further explained that the channel-wise scaling factor and channel-wise bias factor are both obtained from the global conditional vector through two fully connected layers, each containing one hidden layer. The output dimension of each level is the same as the number of channels in the feature map vector at that level. The reason for using affine modulation with multiplication followed by addition instead of directly concatenating the global conditional vector into the feature map vector is that the image change caused by the difference between the target focal position and the axial position of the target reference image is physically manifested as an overall scaling and shift of the original brightness distribution. The mathematical form of affine modulation directly corresponds to this physical process, so that the network does not need to learn this correspondence itself from the data, but only needs to learn the particle-specific deviation.

[0148] Extract the focal space mask corresponding to the target focal position to be generated in the common coordinate system, and perform a pixel-by-pixel multiplication operation on the focal space mask and the initial brightness change matrix to generate the image prediction residual.

[0149] For example, the formula for calculating the image prediction residual is:

[0150] ;

[0151] in, Represents the image prediction residual, in gray levels; This represents the focal space mask corresponding to the target focal position to be generated. Its value is one at the location of the affected pixel and zero at other locations.

[0152] It should be noted that the focal space mask corresponding to the target focal position to be generated is generated in the same way as each acquired frame. That is, the possible defocus range of the focal position is defined by the difference between the axial position of the target focal position and the near and far ends of the candidate focal position. The two-dimensional influence boundary set covering this range is queried from the device axial constraint data and the union of the two-dimensional influence boundary sets is taken. Then, it is mapped from the target base area and the neighboring base areas to the periphery. After multiplying this mask with the initial brightness change matrix pixel by pixel, the image prediction residual of the positions outside the mask is always zero. This means that the generative adversarial network is not allowed to produce any brightness change in areas that are physically impossible to be affected by defocus.

[0153] The image prediction residual is added to the target reference image pixel by pixel and fused to generate a candidate target focal position image.

[0154] Further explanation regarding the training of the pre-constructed generative adversarial network: training samples are taken from a sequence of fully acquired multi-focal images of urine sediment under the same calibrated device. For each particle target, one frame is randomly selected from the sequence as the target reference image, and the axial position of another frame is selected as the target focal position to be generated. The real image of that frame is used as the supervision target. After constructing a multi-channel tensor in the above manner, they are organized in pairs. For example, the sample size is organized with no less than three thousand particle targets and no less than nine axial positions for each particle target. No less than 20% of the particle targets are set as the validation set according to the sample source, and the validation set and the training set do not share the same sample smear. The discriminator adopts the same convolutional feature extraction structure as the generator's encoding end, synchronously inputting the real image and the candidate target focal position image, and inputting the same global conditional vector. The loss function is composed of a weighted sum of adversarial loss and structural penalty loss.

[0155] For example, the formula for calculating structural penalty loss is:

[0156]

[0157] ;

[0158] in, Indicates structural penalty loss; The centroid of the low-frequency subject region in the candidate target focal position image, in pixels; The centroid of the area representing the low-frequency main region in a real image, in pixels; This represents the real image corresponding to the focal position of the target to be generated; Represents the gradient operator; The set of outline pixels representing the base region of the target; This indicates the weight of the low-frequency main body position offset term; The weights of the contour edge splitting terms; The weights of the residual terms representing the internal local light and dark structures; This represents the focal length image of the candidate target.

[0159] It should be further explained that the weights of the three terms are set to ensure that their numerical magnitudes are comparable in the early stages of training. Since the low-frequency subject position offset term is in pixels and its numerical magnitude is usually in the single digits, while the latter two terms are summed along the pixel set and their numerical magnitudes are usually in the range of 10³ to 10⁴, the weight of the low-frequency subject position offset term is set to a range of 100 to 1000, with a typical value of 300. The weight of the contour edge split term is set to a range of 0.1 to 1.0, with a typical value of 0.5. The weight of the internal local light and dark structure residual term is set to a range of 0.01 to 0.1, with a typical value of 0.05. The weighting coefficient between the structural penalty loss and the adversarial loss is set to a range of 1 to 20, with a typical value of 10. The above three measures respectively suppress subject drift, contour splitting, and internal texture fictitiousness during the generation process; an alternating update method is used for training, with the generator weights updated after each update of the discriminator weights, and gradient updates are performed using an adaptive moment estimation optimizer. The learning rate is exemplarily set to 1 x 10^-4 and halved according to the plateau period of the validation set loss; the termination condition is that the distribution of the structural penalty loss on the validation set no longer decreases for three consecutive check periods. The above parameters are given as examples and do not constitute a limitation on the network size or hyperparameter values.

[0160] Through the above technical solution, this embodiment injects the known defocus brightness transformation of the device into the feature map vector in the form of affine modulation, and makes the network only responsible for predicting the brightness change of the particle-specific value. At the same time, the change is zeroed out pixel by pixel using the focal space mask corresponding to the target focal position to be generated. The generation task is decomposed into the part that can be directly given by physical laws and the part that must be learned from data. Unlike the conventional processing method of making the generative adversarial network output a complete target focal position image end-to-end, this embodiment applies a hard spatial constraint derived from the device calibration at the scope of the network output, so that the suppression of artifacts does not depend on the amount of training samples but on structural constraints. The candidate target focal position image obtained in this way is used as the test object for stateful sequence back-verification in subsequent processing.

[0161] Calculate the average value of generated pixels in the target base region within the candidate target focal position image, and then subtract the average background brightness of the corresponding local final net background from the average value of generated pixels before extracting the numerical sign to obtain the generated brightness polarity; calculate the brightness gradient matrix of the target base region within the candidate target focal position image and extract the maximum value to obtain the generated structure state.

[0162] It should be noted that the average background brightness used in the verification is taken from the local final net background of the acquired frame whose axial position is closest to the target focal position to be generated, rather than from the candidate target focal position image itself. The reason for this is that the candidate target focal position image is the generated result. If the average background brightness is calculated based on its own data, the deviation of the generated result will be included in both the detected quantity and the comparison benchmark, causing the comparison to lose its ability to distinguish.

[0163] For example, the calculation formulas for generating light and dark polarities and generating structural states are as follows:

[0164] ;

[0165] ;

[0166] in, This indicates the generation of light and dark polarity; This represents the average generated pixel value of the target's base region within the candidate target focal position image, in gray levels. The frame number represents the acquired frame whose axial position is closest to the focal position of the target to be generated; Indicates the first Average background brightness of the frame; Indicates the generated structure state, in units of gray levels per pixel.

[0167] If the generated light and dark polarity is consistent with the focus contrast polarity, then the value of the generated structural state is lower than that of the particle structure state. If so, the state sequence verification is deemed successful, and the candidate target focal position image is labeled as a physically compliant image.

[0168] To further clarify, each of the two levels of judgment corresponds to an independent optical fact, and the execution order cannot be reversed. The first level judgment determines whether the generated brightness polarity is consistent with the focus contrast polarity. This is based on the fact that, on the same side of the candidate focal position neighborhood, the brightness direction of the grains relative to the background does not change with the degree of defocus. Since the target focal position to be generated is located between two acquired axial positions and does not cross the polarity reversal position within the candidate focal position neighborhood, its brightness direction must be consistent with the adjacent acquired frames. If the generated result shows opposite brightness directions, it indicates that the network has made an incorrect extrapolation across the polarity reversal position. The second level judgment determines whether the generated structural state is lower than the grain structural state. This is based on the fact that, when the target focal position to be generated is further away from the candidate focal position neighborhood relative to the frame corresponding to the target reference image, its contour grayscale transition must be smoother. If the generated structural state is not lower than the grain structural state of the frame corresponding to the target reference image, it indicates that the network has generated a sharper contour in the defocus direction, which is a result that violates the attenuation law of the optical transfer function.

[0169] Additionally, regarding the handling when neither of the two-level judgments is satisfied: if the generated brightness polarity is inconsistent with the focus contrast polarity, or if the generated structural state is not lower than the particle structure state, then the state-based sequence verification fails, and the candidate target focal position image is not labeled as a physically compliant image, nor is it included in the augmented image set; at this time, within the same possible defocus range, a new axial position that does not coincide with the original target focal position to be generated is reassigned, the multi-channel tensor is reconstructed, and a new candidate target focal position image is generated. The upper limit for the number of reassignments is three. After reaching the upper limit, the augmentation of this axial position range is abandoned, and the next particle target is processed instead.

[0170] Through the above technical solution, this embodiment independently extracts the signified brightness direction and contour steepness from the generated result and compares them with the corresponding quantities obtained from actual acquisition in a fixed order of direction first and intensity second. This changes the acceptance criteria of the generated image from visual similarity to conformity to the laws of optical evolution. Unlike the conventional processing method that uses pixel-level error or discriminator scoring as the criteria for generation quality, this embodiment adopts physical quantity comparison independent of the network at the source of verification basis. This ensures that generated results that violate the defocus law are rejected even if they obtain high scores on the discriminator. The resulting physically compliant image is used as the image data ontology for encapsulating traceable augmented samples in subsequent processing.

[0171] Extract the coordinate values ​​of the target focal position to be generated corresponding to the physical compliance image; extract the bound device hardware identifier from the device axial constraint data, and concatenate the coordinate values ​​and the device hardware identifier bit by bit to generate a combined tracking string.

[0172] It should be noted that the coordinate values ​​are the axial position values ​​of the target focal point to be generated, in micrometers, rounded to two decimal places and converted into a fixed-length string; the equipment hardware identifier is the unique equipment number bound to it when calibrating the axial constraint data, and its content includes the equipment model code, the fuselage serial number, and the objective lens configuration code. Bit-by-bit string concatenation means concatenating the coordinate value string and the equipment hardware identifier string end-to-end according to a fixed field order and a fixed field length. If the field length is insufficient, it is padded with leading zeros, thus ensuring that the combined tracking string has a constant length and fixed field boundaries within the same batch of augmented images, facilitating subsequent parsing.

[0173] For example, the calculation expression for the traceable hash check code is as follows:

[0174] ;

[0175] in, Indicates the traceability hash check code; Indicates secure hashing operation; Represents a string of coordinate values; Represents a device hardware identifier string; This indicates a bitwise string concatenation operation.

[0176] It should be noted that the secure hash operation is implemented using a 256-bit secure hash algorithm, and the output is a 64-bit hexadecimal string. The reason for performing a secure hash operation on the combined tracking string instead of directly storing the string is that the augmented image set may be used by multiple parties during subsequent circulation and merging. Directly storing the plaintext field would make it difficult to detect if the field is partially modified, while the hash result is sensitive to any change in any bit of the input and can immediately expose inconsistencies during verification.

[0177] The traceability hash checksum is written as a metadata field into the file header of the physical compliant image, encapsulated to form a traceable augmented sample, and all the encapsulated traceable augmented samples are aggregated to generate an augmented image set.

[0178] Furthermore, in addition to the traceability hash checksum, the metadata fields written to the header of the physically compliant image file also include coordinate values, device hardware identifier, axial position of the frame corresponding to the target reference image, near and far axial positions of the focal position candidate neighborhood, and focus contrast polarity. This allows downstream training tasks to directly parse the defocus attribute of a sample and the optical configuration of the device it is based on when reading the sample. Furthermore, by re-performing a secure hash operation on the coordinate values ​​and device hardware identifier and comparing the results with the stored traceability hash checksum, it can be confirmed that the metadata has not been tampered with.

[0179] It should be noted that the output of the augmented image set is divided into two categories according to the termination situation. The first category is the normal completion situation, in which the output augmented image set contains all traceable augmented samples. The fields of each traceable augmented sample are: image data body, coordinate value, device hardware identifier, axial position of the frame corresponding to the target reference image, near and far axial positions of the focal position candidate neighborhood, focus contrast polarity, and traceable hash check code. For example, the output of a traceable augmented sample is as follows: the image data body is a single-channel grayscale image with the target base area centered; the coordinate value is -1.50 micrometers; the device hardware identifier is a combination of the serial number of a certain model of digital microscopy imaging equipment and the configuration code of a 40x objective lens; the axial position of the frame corresponding to the target reference image is -1.00 micrometers; the near-end axial position of the focal position candidate neighborhood is -2.00 micrometers, and the far-end axial position is 2.00 micrometers; the focus contrast polarity is positive one; the traceability hash check code is a hexadecimal string obtained by concatenating the aforementioned coordinate value string and the device hardware identifier string and then performing a 256-bit secure hash algorithm. The second type is the partial termination case, that is, some particle targets do not generate traceable augmented samples due to continuous failure of state sequence verification, continuous invalidity of temporary net background, or interruption of acquisition sequence. In this case, the output augmented image set only contains the traceable augmented samples that have been encapsulated, and a list of incomplete particle targets is attached. The list fields include the bounding boundary coordinates of the particle targets in the common coordinate system and the termination reason code.

[0180] Additionally, it should be noted that the near and far axial positions of the focal position candidate neighborhood written in the file header are directly taken from the spatial range defined by the axial span feature extraction, while the out-of-focus contrast polarity is directly taken from the difference sign with the local final net background as a reference. An additional beneficial effect of this linkage is that the downstream training task can know the orientation and out-of-focus degree of each traceable augmented sample relative to the reference focal position interval without re-performing any image analysis. This allows for stratified sampling or weighting of the training data according to the out-of-focus attribute, rather than using all augmented samples as indiscriminate homogeneous data.

[0181] Through the above technical solution, this embodiment establishes a verifiable binding relationship between each generated image and the optical calibration of the device it is based on by concatenating the coordinate values ​​of the target focal position to be generated with the device hardware identifier in a fixed-length field, performing a secure hash operation, and writing it into the image file header. Unlike the conventional processing method that only records the generation parameters in the file name or external index table, this embodiment completes parameter binding and integrity verification at the encapsulation level of the sample itself, so that the sample can still independently verify the source of its defocus attribute after cross-database merging and multiple transfers. The traceable augmented sample obtained in this way can be used as training data that can be organized hierarchically according to the defocus attribute in the training stage of the downstream intelligent recognition model.

[0182] Example 2:

[0183] Please see Figure 5 A urine sediment image augmentation system based on generative adversarial networks includes:

[0184] The data acquisition module is used to acquire the axial constraint data of the target object, multifocal images of urine sediment and their corresponding axial positions;

[0185] The common coordinate mapping module is used to extract the position reference of the multifocal urine sediment image and establish a common coordinate system accordingly. Then, the multifocal urine sediment image is mapped to the common coordinate system to generate the target base region and the adjacent base region.

[0186] The focal position range estimation module is used to extract the axial span features of the target base area at each axial position and delineate the focal position candidate neighborhood accordingly. Then, it calculates the difference between the axial position of each frame in the multi-focal urine sediment image and the axial position of the focal position candidate neighborhood to generate the possible defocus range.

[0187] The spatial boundary fusion module is used to query the set of two-dimensional influence boundaries that cover the possible defocus range in the axial constraint data of the equipment, and perform union processing on each two-dimensional influence boundary in a common coordinate system to generate a focal position spatial mask.

[0188] The affected pixel removal module is used to map from the target base region and the adjacent base region to the periphery according to the focal space mask, mark the affected pixels and remove them from the corresponding frame, and then divide the remaining pixels into a set of background sub-regions.

[0189] The local background verification module is used to verify the consistency of local illumination by calculating and comparing the brightness data of each background sub-region, and then combine the verified background sub-regions into the local final net background of the corresponding frame.

[0190] The adversarial augmentation generation module is used to input the local final net background of each frame into a pre-constructed generative adversarial network to generate an augmented image set.

[0191] This embodiment has the same technical effects as Embodiment 1.

[0192] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The data mentioned in this application have undergone normalization and other preprocessing to unify dimensions during formula calculations.

[0193] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A urine sediment image augmentation method based on generative adversarial networks, applied to multi-focal training data augmentation for intelligent recognition of urine sediment microscopic images, characterized in that... Includes the following steps: Acquire the axial constraint data of the target object, multifocal images of urine sediment, and their corresponding axial positions; The location reference of the multifocal urine sediment image is extracted and a common coordinate system is established accordingly. Then, the multifocal urine sediment image is mapped to the common coordinate system to generate the target base region and the adjacent base region. Extract the axial span features of the target base region at each axial position and delineate the focal position candidate neighborhood accordingly. Then calculate the difference between the axial position of each frame in the multi-focal urine sediment image and the axial position of the focal position candidate neighborhood to generate the possible defocus range. Query the set of two-dimensional influence boundaries that cover the possible defocus range in the axial constraint data of the device, and perform union processing on each two-dimensional influence boundary in the common coordinate system to generate a focal position space mask; Based on the focal space mask, the affected pixels are mapped from the target base region and the adjacent base region to the periphery, and removed from the corresponding frame. Then, the remaining pixels are divided into a set of background sub-regions. By calculating and comparing the brightness data of each background sub-region, the consistency of local illumination is verified, and then the verified background sub-regions are combined into the local final net background of the corresponding frame. The local final net background of each frame is input into a pre-constructed generative adversarial network to generate an augmented image set.

2. The urine sediment image augmentation method based on generative adversarial networks according to claim 1, characterized in that: Mapping the multifocal urine sediment image to the common coordinate system to generate the target base region and adjacent base regions specifically includes: Low-pass filtering is performed on each adjacent frame in the multifocal urine sediment image to generate a position reference; Calculate the position coordinate difference between each position reference in adjacent frames and construct a residual objective function accordingly, and then solve for the lateral translation distance that minimizes the residual objective function; Each adjacent frame is translated to the common coordinate system according to the horizontal translation distance to generate an aligned frame, and the particle center position and the relative arrangement topology of the particles formed by each aligned frame are extracted. Calculate the spatial distance difference between the center positions of the particles between the aligned frames, and compare the structural consistency of the relative arrangement topology of each particle; The particle center locations with the smallest spatial distance difference and the best structural consistency are aggregated into a particle location set, and a target base region is generated based on the enclosing boundary of the particle location set. Extract the set of particle positions distributed around the target base region in the common coordinate system to generate the neighboring base region.

3. The urine sediment image augmentation method based on generative adversarial networks according to claim 2, characterized in that: Aggregating the particle center locations with the smallest spatial distance difference and the best structural consistency into a particle location set, and generating the target base region based on the bounding boundary of the particle location set specifically includes: Establish a corresponding mapping for the particle center positions with the smallest spatial distance difference and the structural consistency state showing a matching state, and aggregate the particle center positions with corresponding mappings to generate an initial set of particle positions; The number of corresponding mappings of the initial particle position set between adjacent frames is counted, and the number of corresponding mappings is recorded as the trajectory topology state; When the trajectory topology state exhibits a preset abnormal state, a spatial abnormality event is determined to have occurred; Remove the adjacent frames where the spatial anomaly event occurred from the initial particle location set, and aggregate the particle center positions in the remaining frames after removing the adjacent frames into a particle location set; Extract the extreme values ​​of the outer coordinates of the particle location set in each direction within the common coordinate system to generate an enclosing boundary, and generate the target base region based on the spatial region delineated by the enclosing boundary within the common coordinate system.

4. The urine sediment image augmentation method based on generative adversarial networks according to claim 1, characterized in that: Before defining the candidate neighborhood of the focal point, the process also includes constructing a temporary net background, specifically including: Extract the maximum and minimum values ​​of the axial positions corresponding to each frame and calculate the difference to generate the maximum defocus span; Query the maximum two-dimensional influence boundary corresponding to the maximum defocus span in the axial constraint data of the equipment; In the common coordinate system, a focal position-independent conservative spatial envelope is generated by extending the range defined by the maximum two-dimensional influence boundary outward along the target base region. Pixels located within the focal-independent conservative spatial envelope are removed from the multifocal urine sediment image, and the remaining pixels that are not removed are extracted as a temporary background candidate pixel set. Calculate the mean brightness and standard deviation brightness of the temporary background candidate pixel set, and construct a brightness fluctuation range defined by the mean brightness and the standard deviation brightness; Pixels whose brightness values ​​fall within the brightness fluctuation range are selected as candidate pixels for the temporary background and combined to form a temporary net background.

5. The urine sediment image augmentation method based on generative adversarial networks according to claim 4, characterized in that: Extracting the axial span features of the target base region at each axial position and delineating the candidate neighborhood of the focal position accordingly specifically includes: Calculate the average pixel brightness of the target base region in each frame, and subtract it from the average brightness of the temporary net background. Extract the sign of the subtraction result to generate the brightness polarity state. Calculate the gray-level co-occurrence matrix of the target base region in each frame, and extract the contrast value of the gray-level co-occurrence matrix to generate the internal structure repetition. The brightness polarity state corresponding to each frame is combined with the internal structural repeatability to form an axial span feature; Extract the maximum and minimum values ​​of the brightness of all pixels within the temporary net background and calculate the difference to generate a baseline contrast threshold; Along the sequence direction of the axial position corresponding to each frame, extract a continuous image frame sequence in which the sign of the brightness polarity state is reversed and the internal structure repetition of each frame is greater than the reference contrast threshold. The spatial range is defined based on the axial positions of the beginning and end of the continuous image frame sequence, and a focal position candidate neighborhood is generated.

6. The urine sediment image augmentation method based on generative adversarial networks according to claim 5, characterized in that: The local final net background of each frame is input into a pre-constructed generative adversarial network to generate an augmented image set, specifically including: Calculate the average background brightness of the final net background in the local area, and calculate the difference between the average pixel brightness and the average background brightness. Extract the sign of the difference calculation result to generate the focus contrast polarity. Calculate the grayscale gradient of the target base region in each frame and extract the maximum value to generate the granular structure state; The frame with the largest absolute value of the difference calculation result is selected as the target reference image, and the axial position of the uncollected data within the possible defocus range is specified as the target focal position to be generated. The local final net background of each frame, the target reference image, the target focal position to be generated, the focus contrast polarity, and the grain structure state are combined into a multi-channel tensor, and the multi-channel tensor is input into a pre-constructed generative adversarial network to generate image prediction residuals. The image prediction residual is added to the target reference image pixel by pixel and fused to generate candidate target focal position images. After performing stateful sequence backtesting on the candidate target focal position images, they are aggregated to generate an augmented image set.

7. The urine sediment image augmentation method based on generative adversarial networks according to claim 6, characterized in that: The multi-channel tensor is input into a pre-constructed generative adversarial network to generate image prediction residuals, specifically including: By generating convolutional layers within a generative adversarial network, feature map vectors of the target reference image and the local final net background are extracted respectively. The target focal position to be generated, the focus contrast polarity, and the particle structure state are concatenated along the channel dimension to generate a global conditional vector. The feature map vector is multiplied and added using the global conditional vector to generate an initial brightness change matrix; Extract the focal space mask corresponding to the target focal position to be generated in the common coordinate system, and perform a pixel-by-pixel multiplication operation on the focal space mask and the initial brightness change matrix to generate image prediction residuals.

8. The urine sediment image augmentation method based on generative adversarial networks according to claim 6, characterized in that: Performing stateful sequence backtesting on the candidate target focal position image specifically includes: Calculate the average value of generated pixels in the target base region within the candidate target focal position image, and subtract the average background brightness of the corresponding local final net background from the average value of generated pixels before extracting the numerical sign to obtain the generated brightness polarity. Calculate the brightness gradient matrix of the target base region within the candidate target focal position image and extract the maximum value to obtain the generated structure state; If the generated light and dark polarity is consistent with the focus contrast polarity, then if the value of the generated structural state is lower than the value of the particle structure state, then the state sequence verification is deemed successful, and the candidate target focal position image is labeled as a physically compliant image.

9. The urine sediment image augmentation method based on generative adversarial networks according to claim 8, characterized in that: The generation of augmented image sets specifically includes: Extract the coordinate values ​​of the target focal position to be generated corresponding to the physical compliance image; Extract the bound device hardware identifier from the device axial constraint data, and concatenate the coordinate values ​​with the device hardware identifier bit by bit to generate a combined tracking string; Perform a secure hash operation on the combined tracking string to generate a traceability hash verification code for tracking and location; The traceability hash verification code is written as a metadata field into the file header of the physical compliant image, encapsulated to form a traceable augmented sample, and all the encapsulated traceable augmented samples are aggregated to generate an augmented image set.

10. A urine sediment image augmentation system based on generative adversarial networks, characterized in that, include: The data acquisition module is used to acquire the axial constraint data of the target object, multifocal images of urine sediment and their corresponding axial positions; The common coordinate mapping module is used to extract the position reference of the multifocal urine sediment image and establish a common coordinate system accordingly, and then map the multifocal urine sediment image to the common coordinate system to generate the target base region and the adjacent base region. The focal position range estimation module is used to extract the axial span features of the target base region at each axial position and delineate the focal position candidate neighborhood accordingly. Then, it calculates the difference between the axial position of each frame in the multi-focal urine sediment image and the axial position of the focal position candidate neighborhood to generate the possible defocus range. The spatial boundary fusion module is used to query the set of two-dimensional influence boundaries that cover the possible defocus range in the axial constraint data of the device, and perform union processing on each two-dimensional influence boundary in the common coordinate system to generate a focal position spatial mask. The affected pixel removal module is used to map the focal space mask from the target base region and the adjacent base region to the periphery, mark the affected pixels and remove them from the corresponding frame, and then divide the remaining pixels into a set of background sub-regions. The local background verification module is used to verify the consistency of local illumination by calculating and comparing the brightness data of each background sub-region, and then combine the verified background sub-regions into the local final net background of the corresponding frame. The adversarial augmentation generation module is used to input the local final net background of each frame into a pre-constructed generative adversarial network to generate an augmented image set.

Images