Pet body surface parasite species identification method based on multi-feature data fusion

Abstract

The present application relates to the technical field of image processing, in particular to a pet body surface parasite species identification method based on multi-feature data fusion, first, through the cooperative triggering mechanism, the static image and short video stream of the pet body surface area are synchronously collected, then, three feature extraction channels are executed in parallel: the static morphological feature extraction channel based on deep learning, the dynamic behavior feature extraction channel based on motion trajectory analysis and the skin environment feature extraction channel based on semantic segmentation. Further, a two-level fusion strategy is adopted, and finally the recognition result is output through the classifier. The technical effect of the present application is that the introduction of dynamic behavior characteristics solves the misjudgment problem caused by angle and posture of static image; the combination of skin environment characteristics enhances the diagnosis ability for early or atypical infection, thereby realizing more accurate and reliable automatic parasite species identification.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method for identifying pet ectoparasite species based on multi-feature data fusion. Background Technology

[0002] With the increasing popularity of pet ownership, the prevention and control of pet ectoparasites has become an important issue. Currently, parasite identification mainly relies on visual observation and experience-based judgment by pet owners or veterinarians. This method has significant limitations: on the one hand, different parasites may be similar in morphology, color, and size, making them difficult to distinguish precisely with the naked eye, such as scabies mites and demodectic mites; on the other hand, the dynamic behavioral characteristics of parasites (such as their movement patterns) and the microscopic skin environment characteristics they inhabit (such as the morphology of dandruff and erythema at the parasitic site) are difficult to fully capture and quantify. In recent years, although auxiliary technologies based on single image recognition have emerged, such as analyzing the static morphology of parasites by taking photos of pet skin, these methods are often limited by the shooting angle, lighting conditions, parasite posture, and whether the parasite is in motion, and they ignore the important contextual information of the skin environment on which parasites depend for survival. This results in insufficient accuracy, robustness, and practicality of existing automated identification methods, especially with a high misjudgment rate in early, atypical, or mixed infections. Summary of the Invention

[0003] The purpose of this invention is to provide a method for identifying pet ectoparasite species based on multi-feature data fusion. The introduction of dynamic behavioral features solves the problem of misjudgment caused by angle and posture in static images; the combination of skin environment features enhances the diagnostic ability for early or atypical infections, thereby achieving more accurate and reliable automated parasite species identification.

[0004] To achieve the above objectives, this invention provides a method for identifying pet ectoparasite species based on multi-feature data fusion, comprising the following steps:

[0005] Static images and short-time video streams of the target skin area are simultaneously acquired using image acquisition equipment, and preprocessed separately.

[0006] Three feature extraction channels are executed in parallel: depth morphological feature encoding is extracted from the preprocessed static image, dynamic behavior feature vector is extracted from the preprocessed short video stream, and skin environment pathological feature vector is extracted from the same static image.

[0007] The deep morphological feature encoding and the skin environment pathological feature vector are first-level correlated and fused to generate a morphological environment correlated feature vector; then the morphological environment correlated feature vector is fused with the dynamic behavior feature vector to obtain the category identification result.

[0008] The method further includes:

[0009] Feedback on the identification results of the aforementioned categories is obtained, and incremental learning of the model is performed based on the feedback data to optimize the identification model.

[0010] This includes simultaneously acquiring static images and short-time video streams of the target skin area using image acquisition equipment, including:

[0011] Based on the detection of motion and focus areas in the real-time video stream, when micro-motion and focus are both detected simultaneously and meet the preset conditions, a high-resolution still image is synchronously acquired, and a continuous video containing the time period before and after the triggering time is extracted as a short video stream.

[0012] The extraction of depth morphological feature encoding from the preprocessed static image includes:

[0013] The preprocessed ROI of the insect body is input into a pre-trained and fine-tuned fine-grained deep convolutional neural network for image recognition. The pre-training process of the network includes a first pre-training stage on a general image dataset and a second pre-training stage on a microscopic biological morphology image dataset. The fine-tuning process is performed using a parasite morphology annotation dataset.

[0014] The feature vectors output from the higher layers of the network are extracted as the deep morphological feature encoding.

[0015] The extraction of dynamic behavior feature vectors from the preprocessed short video stream includes:

[0016] The preprocessed dynamic target sequence image stack is used to track moving targets and reconstruct a two-dimensional spatiotemporal motion trajectory.

[0017] Based on the two-dimensional spatiotemporal motion trajectory, motion activity parameters, motion pattern parameters, and motion rhythm parameters are calculated, and these parameters are arranged in order to form the dynamic behavior feature vector.

[0018] Among them, the extraction of skin environmental pathological feature vectors from the same static image includes:

[0019] The skin environment region (ROI) obtained after preprocessing is input into a lightweight semantic segmentation network to obtain pixel-level label maps containing erythema and dandruff regions;

[0020] Based on the pixel-level label map, color and distribution features, texture features and morphological features are quantitatively calculated.

[0021] The calculated feature parameters are combined to form the skin environment pathological feature vector.

[0022] Specifically, the primary association fusion is as follows:

[0023] The deep morphological feature encoding and the skin environment pathological feature vector are projected onto an intermediate semantic space of the same dimension to obtain projected morphological features and projected environment features.

[0024] Calculate the mutual attention weights between the projection morphological features and the projection environment features, and generate attention-weighted morphological features and attention-weighted environment features;

[0025] After concatenating the attention-weighted morphological features and the attention-weighted environmental features, the concatenation is input into a shallow correlation neural network for processing, and the morphological-environment correlation feature vector is output.

[0026] Specifically, advanced decision-making integration includes:

[0027] The dynamic behavior feature vector is projected onto an intermediate semantic space of the same dimension as the feature vector associated with the morphological environment to obtain the projected behavior feature.

[0028] A behavioral feature confidence scalar is calculated based on the dynamic behavioral feature vector, and the projected behavioral features are weighted using this scalar.

[0029] The weighted projection behavior features are added element-by-element to the morphological environment-related feature vector to obtain a comprehensive feature vector;

[0030] The comprehensive feature vector is input into the decision classifier, and the category identification result is output.

[0031] Incremental model learning based on feedback data includes:

[0032] The obtained corrected sample data and its gold standard label are used to construct a feedback training sample package, which is then stored in the incremental learning sample library.

[0033] When the preset triggering conditions are met, data is extracted from the incremental learning sample library and the historical sample replay buffer to form a training batch.

[0034] First, the network weights of each feature extraction channel are frozen, and only the shallow association neural network and decision classifier are trained;

[0035] If the performance improvement does not meet the set requirements, the learning rate is reduced to jointly fine-tune the top-level network weights of each feature extraction channel, the shallow correlation neural network, and the decision classifier.

[0036] Before executing the three feature extraction channels in parallel, the process also includes a step of precise matching and differential preprocessing of the synchronously acquired data:

[0037] High-resolution still images and short-time video streams are identified as paired data based on association identifiers;

[0038] Extract the key video frames that trigger the acquisition from the short video stream, and spatially align them with the high-resolution still image. Correct the entire video stream based on the alignment result.

[0039] The aligned static images were subjected to a first preprocessing for insect morphology analysis and a second preprocessing for skin environment analysis.

[0040] The corrected short-time video stream undergoes a third preprocessing step: temporal stabilization and dynamic target sequence extraction.

[0041] This invention presents a method for identifying pet ectoparasite species based on multi-feature data fusion, aiming to address the problems of low identification accuracy and insufficient diagnostic capability for early and complex infections caused by existing technologies relying on single static image analysis. The method first synchronously acquires static images and short-time video streams of the pet's body surface region through a collaborative triggering mechanism, and performs spatiotemporal alignment and differential preprocessing. Subsequently, three feature extraction channels are executed in parallel: a deep learning-based static morphological feature extraction channel, a motion trajectory analysis-based dynamic behavioral feature extraction channel, and a semantic segmentation-based skin environment feature extraction channel. Furthermore, a two-level fusion strategy is employed: first, morphological and environmental features are initially fused through an interactive attention mechanism to generate joint semantic features; then, these features are fused with dynamic behavioral features through a confidence-weighted high-level decision fusion, and finally, the identification result is output through a classifier. In addition, the method also includes an incremental learning optimization mechanism based on expert feedback. The technical advantages of this invention are as follows: through the effective complementarity and deep fusion of multi-dimensional heterogeneous features, the recognition accuracy and robustness to complex situations are significantly improved; the introduction of dynamic behavioral features solves the problem of misjudgment caused by angle and posture in static images; and the combination of skin environment features enhances the diagnostic ability for early or atypical infections, thereby achieving more accurate and reliable automated parasite species identification. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.

[0043] Figure 1 This is a schematic diagram illustrating the steps of a method for identifying pet ectoparasite species based on multi-feature data fusion provided by the present invention.

[0044] Figure 2 This is a schematic diagram illustrating the complete steps of a method for identifying pet ectoparasite species based on multi-feature data fusion, provided by the present invention.

[0045] Figure 3 This is a flowchart illustrating a method for identifying pet ectoparasite species based on multi-feature data fusion, provided by the present invention. Detailed Implementation

[0046] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application.

[0047] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0048] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0049] Please see Figures 1-3 This invention provides a method for identifying pet ectoparasite species based on multi-feature data fusion, comprising the following steps:

[0050] S101. Simultaneously acquire static images and short video streams of the target skin area using an image acquisition device, and preprocess them separately.

[0051] Specifically, to solve the problem of synchronous acquisition, this invention adopts a "master-slave collaborative triggering" acquisition mechanism. The preferred image acquisition device is a smart skin mirror with high frame rate video streaming and high-speed image capture capabilities.

[0052] Main data stream (dynamic video): The device is in real-time video preview mode by default, continuously capturing video stream data at a high frame rate (such as 30fps) and caching it in a circular buffer.

[0053] Data-triggered (high-precision still images): The device's built-in or connected processor runs a lightweight motion and focus area detection algorithm in real time, analyzing the video preview frames. When the algorithm detects a skin area under the lens that simultaneously meets the following two conditions, it automatically triggers a "capture" command:

[0054] Significant micro-movements: Pixel-level motion trajectories consistent with the movement characteristics of small parasites were detected.

[0055] Sharp focus area: This moving area is simultaneously within the sharp focus range determined by the device's autofocus system.

[0056] After the trigger command is issued, the system performs atomic operations:

[0057] From the data: immediately control the camera sensor to perform a full-resolution exposure, generating a high-resolution still image.

[0058] Master data: Simultaneously, extract continuous video data (approximately 5 seconds in total) from approximately 2 seconds before the trigger moment to approximately 3 seconds after the trigger moment from the video circular buffer, as a short video stream corresponding to the still image.

[0059] Spatiotemporal stamping: The system automatically assigns the same set of high-precision timestamps and device spatial location identifiers to this set of data (a still image + a video stream).

[0060] After acquisition, the two types of data (static image files and video stream files) are transmitted to the processing unit (such as a smartphone, edge computing box, or cloud) via the device's internal bus or high-speed interface (such as USB 3.0 or Wi-Fi 6). To avoid logical conflicts and mismatches during transmission, this invention adopts a "digital tag binding and channel-specific ordered transmission" strategy.

[0061] When generating a data set, the system not only adds a timestamp but also generates a unique associated identifier for that data set. This identifier will be written to the metadata (EXIF information) of still image files and at the beginning of video stream files or in a dedicated supplementary file.

[0062] The device has two logical transmission channels internally:

[0063] High-priority channel: Used to transmit relatively small amounts of high-resolution still images and their metadata (including associated identifiers) that require immediate processing.

[0064] Streaming buffer channel: Used for continuous transmission of short video stream data blocks. The same association identifier is also embedded in the header of each video data block.

[0065] Based on the associated identifiers, the arriving static images and video stream data blocks are paired and reassembled to restore complete data groups. Since the identifiers are unique and the size of the data groups is controllable, this process has low computational overhead and almost no matching conflicts or misalignments.

[0066] Upon receiving a data set, it is first identified as paired data based on the association identifier. Then, a "spatiotemporal anchor alignment" operation is performed.

[0067] Extract the video frame that triggered the capture command from the short video stream (i.e., the key video frame).

[0068] High-resolution still images are registered with key video frames using feature point matching algorithms (such as lightweight versions of SIFT or ORB). A spatial transformation matrix (such as an affine transformation) is calculated, which describes the offset, rotation, or scaling caused by minute device jitter within a very short time interval.

[0069] Using this transformation matrix, spatial alignment correction is performed on all frames of the entire short video stream, achieving pixel-level alignment with the high-resolution still image in terms of both field of view and content. Thus, the two images and the video segment achieve precise "referring to the same area" in terms of content.

[0070] Preprocessing for high-resolution still images:

[0071] Precise target region localization and cropping: On the aligned static image, based on the coordinates recorded when motion detection is triggered, the suspected parasite region is initially located. The core is to distinguish between genuine parasite activity and common disturbances (such as hair movement), as detailed below:

[0072] Motion Feature Screening: After detecting pixel-level motion, the system first analyzes its motion pattern. The movement of parasites (such as fleas and mites) typically manifests as continuous, directional micro-displacements or regular wriggling. In contrast, hair movement often appears as passive, random low-frequency swaying or large-scale overall drifting. The algorithm initially screens out suspicious targets by analyzing the continuity of the motion trajectory, directional consistency, and displacement.

[0073] Morphological and textural analysis: Within the movement area, the system simultaneously performs micro-textural analysis. Parasites possess relatively fixed and dense morphological structures and specific textures (such as limb outlines). In contrast, hair appears as long, thin, bright lines with a uniform texture under a microscopic level. By comparing the morphological compactness, edge sharpness, and textural features of the movement area, it is possible to distinguish between parasites and hair.

[0074] Spatiotemporal context verification: The system employs short-sequence analysis to observe the movement's behavior across several consecutive frames. Real parasite activity typically exhibits a recurring pattern of continuous or intermittent occurrences in time. In contrast, fur swaying is often irregular, instantaneous, or occurs alongside large movements of the pet, differing from the relative movement pattern of the skin.

[0075] Then, a salient object detection network is further employed to accurately segment the parasite body and its adjacent skin background within the region, generating two sub-images: Region 1 (parasite body ROI) and Region 2 (skin environment ROI). This avoids the computational waste of full-image processing and separates the focus of attention for different feature channels in subsequent processing.

[0076] Multi-scale detail enhancement: For Region 1 (ROI) of the insect body, adaptive sharpening and local contrast enhancement algorithms are used to highlight the texture of key morphological details such as the insect's outline, limbs, and mouthparts. For Region 2 (ROI) of the skin environment, the focus is on color constancy correction to restore the true colors of erythema and dandruff, and multi-directional filtering is applied to enhance the display of skin lesion texture.

[0077] Standardization: The two ROI sub-images are scaled to a fixed size required by the feature extraction network and the pixel values ​​are normalized.

[0078] Preprocessing for short video streams:

[0079] Temporal stabilization and denoising: Despite spatial alignment, a video stabilization algorithm based on feature point trajectories is used to eliminate inter-frame jitter caused by pet breathing or slight tremors, ensuring that the observed target is relatively stable in the frame.

[0080] Dynamic target sequence extraction: Using background subtraction or optical flow, foreground targets (i.e., parasites) moving near the ROI (Region of Interest) of the parasite are continuously tracked and segmented in an aligned and stable video sequence, generating a "dynamic target sequence image stack" composed of target slices from consecutive frames. This stack is the direct input for subsequent behavioral feature analysis.

[0081] Illumination consistency adjustment: Perform temporal illumination equalization on video sequences to reduce target brightness flicker caused by changes in device or ambient light, ensuring that behavior analysis is not affected by illumination.

[0082] S102. Parallel execution of three feature extraction channels: extracting depth morphological feature encoding from the preprocessed static image, extracting dynamic behavior feature vectors from the preprocessed short video stream, and extracting skin environment pathological feature vectors from the same static image.

[0083] Specifically, after data acquisition and precise preprocessing, the goal of this step is to extract complementary and informative feature vectors from the preprocessed data using three parallel, specialized feature extraction channels. These three channels start synchronously, process their respective preprocessing results, and ultimately output feature vectors in a unified format to prepare for subsequent fusion decisions.

[0084] Channel 1: Static Morphological Feature Extraction Channel. This channel is dedicated to extracting microscopic morphological features from Region 1 (ROI) of the worm in high-resolution static images that can accurately characterize the differences between parasite species.

[0085] Core network architecture and pre-training process:

[0086] A modified fine-grained image recognition deep convolutional neural network is used as the backbone network. Instead of being trained from scratch during the pre-training phase, this network employs a two-stage transfer learning strategy:

[0087] General visual feature pre-training: The network is first pre-trained on a very large-scale general image dataset to learn the ability to extract basic visual features such as edges, corners, textures, and shapes. This stage enables the network to have a strong foundation for general image understanding.

[0088] Adaptive Pre-training for the Microscopic Biological Morphology Domain: To further adapt the network to the specific domain of "microscopic biological morphology recognition," the pre-trained network was subjected to a second stage of pre-training and fine-tuning on a large open-source microscopic image dataset. This dataset contains images of various microscopic targets such as microorganisms, cells, and insect limbs. The key to this stage is to enable the network to learn to focus on and extract discriminative features at the microscopic scale, such as surface textures, jagged edges, and proportional relationships of specific parts, thereby completing the domain transfer from "general vision" to "microscopic morphology perception."

[0089] Specific fine-tuning and feature extraction process for parasite morphology:

[0090] The network, which underwent two stages of pre-training, was then fine-tuned on the parasite morphology annotation dataset constructed in this system. This dataset consists of numerous Region of Interest (ROI) images of parasites, each with precisely labeled boundaries and species by experts. During this stage, the last few fully connected layers of the network were replaced and retrained to learn how to map extracted microscopic morphological features to specific parasite categories. More importantly, the feature extraction capabilities of the network's intermediate layers were specifically optimized, making them exceptionally sensitive to key distinguishing features such as parasite arthropod structure, mouthpart morphology, and body hair distribution.

[0091] During forward inference, when the preprocessed ROI (Region of Interest) of the worm (Region 1) is input into the fine-tuned network, we do not directly use the output of the network's final classification layer. Instead, we extract the feature map output after the last convolutional layer and before the global pooling layer, or the high-dimensional feature vector before the last fully connected layer. This high-dimensional vector is called deep morphological feature encoding. It contains highly abstracted morphological semantic information implied by the input image after multiple nonlinear transformations of the network, and is a powerful and compact mathematical representation of the overall and local morphology of the worm.

[0092] A progressive two-stage transfer learning and fine-tuning strategy is adopted, progressing from "general vision → microscopic biological morphology → parasite specialization". This enables the network to not only possess strong basic feature extraction capabilities but also deeply adapt to the specific needs of microscopic parasite morphology recognition. The extracted deep morphological feature encodings are more discriminative and robust than models that directly use general models or are trained only on small parasite datasets.

[0093] Channel 2: Dynamic Behavioral Feature Extraction Channel. The task of this channel is to quantitatively analyze and extract the movement behavior features of parasites from the dynamic target sequence image stack.

[0094] Detailed extraction process:

[0095] For the input video sequence stack (which is already stable and aligned), an algorithm combining an improved optical flow method and Kalman filter prediction is used to continuously track the moving target (parasite) segmented within each frame of the stack. The algorithm not only calculates the displacement of the target between adjacent frames, but also smooths the trajectory through a prediction-correction mechanism, effectively dealing with situations where the target is briefly occluded or interfered with by image noise, and finally outputs a smooth and continuous two-dimensional spatiotemporal motion trajectory.

[0096] Based on the reconstructed two-dimensional spatiotemporal motion trajectory, a series of well-defined behavioral parameters are calculated to form a dynamic behavioral feature vector:

[0097] Motion activity parameters include average movement speed, standard deviation of speed (characterizing the characteristics of movement that fluctuates in speed), and total movement path length.

[0098] Motion mode parameters: calculate the curvature change of the trajectory, distinguish between linear motion, detour motion, or stationary rotation; analyze the temporal changes of the motion direction, and extract its turning frequency and angle distribution.

[0099] Motion rhythm parameters: By analyzing time series signals of velocity or displacement, the presence of periodic pauses or bursts of motion rhythm can be detected.

[0100] Eigenvectorization: The scalar parameters (such as average velocity, turning frequency, etc.) obtained from the above calculations are arranged in an ordered manner into a fixed-dimensional numerical vector, namely the dynamic behavior feature vector. This vector objectively and quantitatively describes the target's motion "fingerprint" within the observation time window.

[0101] This transforms behavioral analysis from qualitative observation to quantitative computation. A robust tracking algorithm combining optical flow and filtered prediction ensures accuracy in trajectory reconstruction within complex microscopic scenarios. The designed multi-dimensional behavioral parameter quantification system systematically covers the speed, pattern, and rhythm of movement, effectively distinguishing key behavioral differences such as "rapid jumping" (fleas), "slow crawling" (lice), and "intermittent wriggling" (mites).

[0102] Channel 3: Skin Environment Feature Extraction Channel. This channel aims to extract contextual features from Region 2 (ROI) of the skin environment in high-resolution static images to characterize the host skin response and pathological changes caused by parasite infestation.

[0103] Detailed extraction process:

[0104] First, a trained lightweight semantic segmentation network is used to process the ROI image. This network is able to identify and segment different components in the image, outputting a pixel-level label map that clearly identifies erythematous areas, dandruff areas, healthy skin areas, hair areas, and suspected scabs or papules.

[0105] Based on the above segmentation results, multi-dimensional and quantifiable feature calculations are performed:

[0106] Color and distribution characteristics: Calculate the average chromaticity value, saturation, area ratio, and spatial distribution dispersion of the erythematous region within the ROI. Calculate the morphology (e.g., flaky, powdery) and density of the dandruff region.

[0107] Texture features: Multi-scale filter banks were applied to healthy skin areas and erythematous areas respectively to extract their texture spectrum features, quantifying the differences in texture roughness and directionality between pathological skin and healthy skin.

[0108] Morphological characteristics: For the segmented areas suspected of crusting or papules, calculate their number, average size, roundness and other morphological parameters.

[0109] Hair condition characteristics: Analyze the density, breakage rate and degree of disorder in the hair area.

[0110] All the statistical parameters obtained from the above calculations, such as color, texture, shape, and distribution, are combined in a preset order to form a comprehensive skin environmental pathological feature vector. This vector transforms the "on-site evidence" of parasite infestation—the skin's reaction pattern—into a series of numerical indicators that can be processed by machine learning models.

[0111] This approach elevates the skin environment from background information to a crucial diagnostic context. Through a multi-scale feature quantification method guided by semantic segmentation, not only are macroscopic color and texture extracted, but the morphological and spatial distribution characteristics of skin lesions are also quantified in greater detail. This transforms the visual diagnostic experience of veterinarians (such as "tunnel-like" lesions suggesting scabies mites) into calculable and comparable quantitative features, providing powerful auxiliary and corroborating information for parasite identification.

[0112] The three channels operate independently, receiving the ROI (Region of Interest) of the insect body (Region 1), the dynamic target sequence image stack, and the ROI of the skin environment (Region 2) from the preprocessing stage as inputs, respectively. The system uses a unified association identifier to ensure that the three channels process data from the same detection event. They start almost simultaneously, utilizing the multi-core parallel computing capabilities of modern processors to synchronously complete feature extraction, ultimately outputting deep morphological feature codes, dynamic behavioral feature vectors, and skin environment pathological feature vectors, awaiting subsequent fusion.

[0113] S103. Perform a primary correlation fusion between the deep morphological feature encoding and the skin environment pathological feature vector to generate a morphological environment correlation feature vector; then perform a high-level decision fusion between the morphological environment correlation feature vector and the dynamic behavior feature vector to obtain the category identification result.

[0114] Specifically, the system utilizes deep morphological feature encoding from channel one (assuming a dimension of M) and skin environmental pathological feature vectors from channel three (assuming a dimension of S). Since the two feature vectors originate from networks with different structures and quantization methods, their numerical distributions and semantic scales differ. First, batch normalization is applied to both vectors to achieve a mean of 0 and a standard deviation of 1, thus aligning their initial numerical ranges. Then, two independent fully connected layers (which can be viewed as feature projection networks) project the M-dimensional and S-dimensional features onto a common, pre-defined intermediate semantic space dimension D (e.g., 256 dimensions). The projected features are termed projected morphological features and projected environmental features, respectively. This step ensures that subsequent fusion is performed within a unified feature space.

[0115] The attention weights of the projected morphological features on each element of the projected environmental features are calculated. This means the system learns that "for the specific morphology of this parasite, which skin reaction pattern (such as severe erythema or excessive dandruff) is more likely to be relevant evidence." Simultaneously, the attention weights of the projected environmental features on each element of the projected morphological features are calculated. This is equivalent to learning "in the current dermatopathological context, which morphological details of the parasite (such as mouthpart type or body hair) deserve more attention." Based on their respective attention weights, the original projected features are reconstructed using weighted methods to obtain attention-weighted morphological features and attention-weighted environmental features. At this point, each feature has been infused with contextual guidance information from the other feature.

[0116] These two D-dimensional vectors are concatenated to form a 2D joint vector. This joint vector is then input into a shallow relational neural network (typically 2-3 fully connected layers). The network's task is to learn complex non-linear association rules between morphological and environmental features, such as the strong association between "a certain mouthpart morphology" and "tunnel-like skin lesions." The final output is a D-dimensional morphological-environmental association feature vector. This vector is no longer a simple juxtaposition of morphological and environmental features, but rather a novel and compact feature representation that encapsulates the deep semantic association between the two.

[0117] The dynamic behavior feature vector of Channel 2 (assuming a dimension of B) consists of explicit physical parameters, and its semantics are relatively clear, but there may be gaps with the high-level semantic space. First, it is also batch-normalized. Then, it is projected onto the same intermediate semantic space dimension D as in the first stage through a fully connected layer to form projected behavior features. In addition, a lightweight behavior saliency estimator is introduced to generate a behavior feature confidence scalar based on the statistical properties of the behavior features themselves (such as the confidence of movement speed and the completeness of the trajectory), which is used for subsequent weighting.

[0118] The projected behavioral features are weighted using a scalar of confidence scores as weights. Then, the weighted behavioral features are fused with the morphology-environment association feature vector element-wise (rather than concatenated). This addition operation encourages the network to integrate and compensate feature elements in a unified D-dimensional space. For example, when static features are ambiguous, high-confidence dynamic behavioral features can play a decisive role. The fused D-dimensional comprehensive feature vector is then input into the final decision classifier. This classifier is a multilayer perceptron, with its last layer being a softmax layer, and the number of neurons equal to the number of parasite species to be identified. The decision classifier outputs a species probability distribution vector, where each element represents the probability that the input sample belongs to a specific parasite category.

[0119] S104. Obtain feedback on the identification results of the aforementioned categories, and perform incremental model learning based on the feedback data to optimize the identification model.

[0120] Specifically, after the system outputs the probability distribution vector of species and provides the final identification suggestion during daily use, veterinary experts are allowed to review it on the interactive interface.

[0121] Feedback actions: Experts can perform two actions:

[0122] Confirmation: The system's identification result is acknowledged.

[0123] Correction: Select the correct species label from the list, or mark it as "unknown / suspected new species".

[0124] High-value sample inclusion: Once an expert performs a "correction" operation or "confirms" a low-confidence identification result from the system, the current sample is determined to have high training value. The system will automatically trigger the sample data package archiving process.

[0125] The raw data (high-resolution still images, short video streams) associated with the identification event and its preprocessed intermediate data (parasite body region 1 ROI, skin environment region 2 ROI, dynamic target sequence image stack) are stored in encrypted form.

[0126] The corrected label provided by the experts serves as the gold standard label for this sample.

[0127] The above data, preprocessing results, and gold standard labels together constitute a "feedback training sample package," which is stored in an independent incremental learning sample library. Each sample package retains its original association identifier to ensure the integrity of the data chain.

[0128] The model is not updated in real time, but is triggered when specific conditions are met to ensure stability and training efficiency.

[0129] Triggering conditions: The system periodically checks the incremental learning sample library. When any of the following conditions are met, an incremental learning (model optimization) process is automatically initiated:

[0130] Quantity trigger: The number of feedback training sample packages accumulated in the library reaches a preset threshold (e.g., 100).

[0131] Time-triggered: Reaching a preset periodic update time (e.g., every two weeks).

[0132] Diversity trigger: The system detects a significant increase in "corrected" samples belonging to a specific category among newly accumulated samples, or the appearance of "suspected new species" markers, indicating that the model needs urgent enhancement in that category or boundary.

[0133] Two-stage progressive fine-tuning training strategy: To ensure the stability of incremental learning and avoid "catastrophic forgetting," a phased training strategy is adopted:

[0134] Phase One: Strengthening Connections and Decision-Making Levels

[0135] New sample packets are taken from the incremental learning sample library, and a portion of historical training data (data in the historical sample replay buffer) is randomly extracted to form the batch data for this training.

[0136] The network weights of the three feature extraction channels (static morphology, dynamic behavior, and skin environment) are frozen to maintain their strong basic feature extraction capabilities.

[0137] Only the shallow association neural network and decision classifier from step three are unlocked and trained. At this stage, the system learns how to better establish new morphological-environment association rules based on existing basic features and adjust the final classification decision boundary. This is equivalent to allowing the system to quickly absorb new diagnostic experience.

[0138] Phase Two: Adaptive Fine-tuning of the Feature Extraction Layer

[0139] If the model's performance improvement on the independent validation set does not meet expectations after Phase 1 training, or if new samples exhibit unprecedented features (such as entirely new skin pathology patterns), then this phase is initiated.

[0140] Unlock the last few layers of the network for the three feature extraction channels.

[0141] Using all training data (new samples + historical replay data), the top layers of the unlocked association network, decision classifier, and feature extraction channels are jointly fine-tuned at an extremely low learning rate. This allows the system's underlying feature extractor to perform more refined self-tuning to adapt to the new distribution in the feature space.

[0142] After incremental learning training is complete, the new model must be evaluated on a strictly confidential, independent validation set that does not overlap with the training data. Evaluation metrics include not only overall accuracy but also, in particular, improvements in the recognition performance of categories or sample types that have been corrected by experts.

[0143] Newly evaluated models will be saved as a new model version (e.g., V2.1) and archived along with complete training logs and a list of sample packages used. Older model versions will be kept as backups for quick rollback in case of unexpected degradation in the new version.

[0144] Once the performance of the new model version is confirmed to be stable, the system will perform a seamless hot update of the model online. New recognition requests will be handled by the new model version, and the entire update process will be seamless for end users, ensuring service continuity.

[0145] It can achieve the following significant effects:

[0146] Improve recognition accuracy and robustness: By integrating multi-dimensional features such as static morphology, dynamic behavior and skin environment, especially by introducing short-time video stream analysis of behavioral features, it can overcome misjudgments caused by angle, lighting and posture of a single static image, and significantly improve the ability to distinguish similar parasite species.

[0147] Achieving early and complex infection identification: The introduction of skin environmental characteristics (such as skin lesion patterns and hair condition) enables auxiliary judgment through abnormalities in the parasitic environment when the parasite itself is not obvious or is in an early stage, which is beneficial for early warning and diagnosis of mixed infections.

[0148] Enhancing system usability and automation: The solution design considers the entire process from data acquisition and feature extraction to fusion decision-making, forming a closed loop, and introduces an optimization mechanism based on expert feedback, enabling the system to continuously learn and adapt to new cases, reducing over-reliance on the professional experience of operators, and improving the reliability of automated diagnosis.

[0149] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0150] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope.

Claims

1. A pet ectoparasite species identification method based on multi-feature data fusion, characterized in that, Includes the following steps: Static images and short-time video streams of the target skin area are simultaneously acquired using image acquisition equipment, and preprocessed separately. Three feature extraction channels are executed in parallel: depth morphological feature encoding is extracted from the preprocessed static image, dynamic behavior feature vector is extracted from the preprocessed short video stream, and skin environment pathological feature vector is extracted from the same static image. The deep morphological feature encoding and the skin environment pathological feature vector are first-level correlated and fused to generate a morphological environment correlated feature vector; then the morphological environment correlated feature vector is fused with the dynamic behavior feature vector to obtain the category identification result. The primary association fusion is as follows: The deep morphological feature encoding and the skin environment pathological feature vector are projected onto an intermediate semantic space of the same dimension to obtain projected morphological features and projected environment features. Calculate the mutual attention weights between the projection morphological features and the projection environment features, and generate attention-weighted morphological features and attention-weighted environment features; After concatenating the attention-weighted morphological features and the attention-weighted environmental features, the concatenation is input into a shallow correlation neural network for processing, and the morphological-environment correlation feature vector is output. Advanced decision integration specifically includes: The dynamic behavior feature vector is projected onto an intermediate semantic space of the same dimension as the feature vector associated with the morphological environment to obtain the projected behavior feature. A behavioral feature confidence scalar is calculated based on the dynamic behavioral feature vector, and the projected behavioral features are weighted using this scalar. The weighted projection behavior features are added element-by-element to the morphological environment-related feature vector to obtain a comprehensive feature vector; The comprehensive feature vector is input into the decision classifier, which outputs the category identification result. Extracting depth morphological feature encoding from the preprocessed static image includes: The pre-processed ROI of the insect body is input into a pre-trained and fine-tuned fine-grained deep convolutional neural network for image recognition. The pre-training process of the network includes a first pre-training stage on a general image dataset and a second pre-training stage on a microscopic biological morphology image dataset. The fine-tuning process is performed using a parasite morphology annotation dataset. The feature vectors output from the higher layers of the network are extracted as the deep morphological feature encoding. Extract dynamic behavior feature vectors from the preprocessed short video stream, including: The preprocessed dynamic target sequence image stack is used to track moving targets and reconstruct a two-dimensional spatiotemporal motion trajectory. Based on the two-dimensional spatiotemporal motion trajectory, motion activity parameters, motion pattern parameters, and motion rhythm parameters are calculated, and these parameters are arranged in order to form the dynamic behavior feature vector. Extracting skin environmental pathological feature vectors from the same static image, including: The skin environment region (ROI) obtained after preprocessing is input into a lightweight semantic segmentation network to obtain pixel-level label maps containing erythema and dandruff regions; Based on the pixel-level label map, color and distribution features, texture features and morphological features are quantitatively calculated. The calculated feature parameters are combined to form the skin environment pathological feature vector. 2.The pet ectoparasite species identification method based on multi-feature data fusion according to claim 1, wherein, The method further includes: Feedback on the identification results of the aforementioned categories is obtained, and incremental learning of the model is performed based on the feedback data to optimize the identification model.

3. The method for identifying pet ectoparasite species based on multi-feature data fusion as described in claim 1, characterized in that, The image acquisition device simultaneously acquires static images and short video streams of the target skin area, including: Based on the detection of motion and focus areas in the real-time video stream, when micro-motion and focus that meet the preset conditions are detected at the same time, a high-resolution still image is synchronously triggered and a continuous video containing the time period before and after the trigger time is extracted as a short video stream.

4. The method for identifying pet ectoparasite species based on multi-feature data fusion as described in claim 2, characterized in that, Incremental model learning based on feedback data includes: The obtained corrected sample data and its gold standard label are used to construct a feedback training sample package, which is then stored in the incremental learning sample library. When the preset triggering conditions are met, data is extracted from the incremental learning sample library and the historical sample replay buffer to form a training batch. First, the network weights of each feature extraction channel are frozen, and only the shallow association neural network and decision classifier are trained; If the performance improvement does not meet the set requirements, the learning rate is reduced to jointly fine-tune the top-level network weights of each feature extraction channel, the shallow correlation neural network, and the decision classifier.

5. The method for identifying pet ectoparasite species based on multi-feature data fusion as described in claim 1, characterized in that, Before executing the three feature extraction channels in parallel, the process also includes a step of precise matching and differential preprocessing of the synchronously acquired data: High-resolution still images and short-time video streams are identified as paired data based on association identifiers; Extract the key video frames that trigger the acquisition from the short video stream, and spatially align them with the high-resolution still image. Correct the entire video stream based on the alignment result. The aligned static images were subjected to a first preprocessing for insect morphology analysis and a second preprocessing for skin environment analysis. The corrected short-time video stream undergoes a third preprocessing step: temporal stabilization and dynamic target sequence extraction.

Images