Pet Search Geographic Information Matching Method Based on Machine Learning and Natural Language Processing

Abstract

This application relates to a pet search geographical information matching method based on machine learning and natural language processing. The method includes: obtaining a pet image, a text description, and lost spatio-temporal information input by a user, where the lost spatio-temporal information includes lost location coordinates and lost time; extracting visual features of the pet image and semantic features of the text description through a deep learning model; encoding the lost spatio-temporal information into a spatio-temporal joint vector, and fusing the visual features and semantic features to calculate a dynamic matching degree; generating a visual heat map according to the dynamic matching degree, and screening through comparison with a preset screening threshold to output an interactive map interface. This method comprehensively analyzes visual, semantic, and spatio-temporal information through machine learning and natural language technologies, significantly improving the accuracy and efficiency of pet search, solving the problems of insufficient information utilization and low matching accuracy in traditional pet search methods, and providing strong support for quickly retrieving lost pets.

Description

Technical Field

[0001] The present invention belongs to the field of computer technology, and particularly relates to a lost pet geographical information matching method based on machine learning and natural language processing. Background Art

[0002] With the rapid development of technologies in the field of artificial intelligence, in the field of pet searching, there has emerged a pet searching platform technology based on information technology. This technology can attempt to match relevant records of lost pets and found pets by integrating information provided by users. Compared with the traditional offline pet searching method, it breaks through the limitations of time and space and can quickly spread pet searching information over a larger range. However, the existing pet searching platforms can only provide a platform for users to upload pet searching notices. On the one hand, the pet owner usually can only search for relevant information by keywords such as the approximate lost location. When the description entered by the user does not exactly match the description of the publisher, it is very difficult to accurately match the relevant missing pet information. On the other hand, the lost pet information uploaded by users is not processed in a timely manner, resulting in a waste of a large amount of effective information, and the information update speed is slow, making many of the information retrieved by users may have become outdated. For example, the pet may have been retrieved but the information still shows as lost on the platform, resulting in users having to search for information related to their pets manually in a vast amount of information, which not only consumes a large amount of time and energy but also is prone to omissions and errors, leading to a low success rate of pet searching. Summary of the Invention

[0003] Based on this, in view of the above technical problems, it is necessary to provide a lost pet geographical information matching method based on machine learning and natural language processing to improve the efficiency and accuracy of pet searching and enhance the reliability of predicting the activity range of lost pets.

[0004] In a first aspect, the present application provides a lost pet geographical information matching method based on machine learning and natural language processing, including:

[0005] Obtain a pet image, a text description, and lost spatio-temporal information input by a user, where the lost spatio-temporal information includes a lost location coordinate and a lost time;

[0006] Extract visual features of the pet image and semantic features of the text description through a deep learning model, where the visual features include breed, color, body shape, and special marks, and the semantic features include breed entities, color modifiers, and behavioral features;

[0007] Encode the lost spatio-temporal information into a spatio-temporal joint vector, and fuse the spatio-temporal joint vector, visual features, and semantic features to calculate a dynamic matching degree;

[0008] Predict the activity range of pets according to the dynamic matching degree and generate a visual heat map, and screen and sort the dynamic matching degree based on the visual heat map and the preset screening threshold, and output an interactive map interface.

[0009] In one embodiment, the visual features of the pet image and the semantic features of the text description are extracted through a deep learning model, including:

[0010] Adopt the Vision Transformer model to divide the pet image into multiple pixel blocks, perform linear projection on each pixel block to generate initial image features, and optimize the initial image features based on a multi-level self-attention mechanism to generate optimized image features;

[0011] Perform multi-instance weak supervision learning on the optimized image features, and use the MS-Captivator model to generate structured pet labels. The structured pet labels include breed classification codes, color spectrum vectors, body shape topologies, and special marker position matrices;

[0012] Perform multi-head self-attention dynamic weighting on the structured pet labels, calculate the weights of each feature channel through the following formula, and output visual features:

[0013] ;

[0014] Where Q is the query vector, is the key vector of the i-th feature channel, and d is the feature dimension;

[0015] Input the text description into the RoBERTa model for semantic parsing to generate initial semantic features, and use the structured pet labels as cross-modal prior knowledge. Through the cross-modal cross-attention mechanism, fuse the visual features with the initial semantic features to generate enhanced semantic features;

[0016] Based on the enhanced semantic features, calculate the difference loss between the color spectrum vector and the color description in the text description through the color semantic calibration network, and backpropagate to optimize the parameters of the MS-Captivator model and the RoBERTa model, and output semantic features strongly associated with the visual features.

[0017] In one embodiment, encode the lost spatio-temporal information into a spatio-temporal joint vector, including:

[0018] Taking the lost position coordinates as the center point, calculate the Gaussian kernel encoding of the geographical coordinates through the following formula:

[0019] ;

[0020] Where σ = 0.1 is the sensitivity coefficient, which is obtained through training of historical data, is the lost position coordinate;

[0021] Calculate the attenuation factor according to the lost time through the following calculation formula:

[0022] ;

[0023] t is the lost time, and 48 is the reference attenuation period;

[0024] Concatenate the Gaussian kernel encoding and the attenuation factor to generate a spatio-temporal feature vector, and normalize the spatio-temporal feature vector to obtain a spatio-temporal joint vector.

[0025] In one embodiment, fuse the visual feature, semantic feature and spatio-temporal joint vector, and calculate the dynamic matching degree, including:

[0026] Obtain the candidate animal image, and extract the features of the candidate animal image through a deep learning model to obtain the visual feature of the candidate animal image;

[0027] Calculate the cosine similarity between the visual feature and the visual feature of the candidate animal image;

[0028] Calculate the semantic similarity between the semantic feature and the text description of the candidate animal through the BERTScore algorithm;

[0029] Perform weighted fusion on the spatio-temporal joint vector, cosine similarity and semantic similarity to obtain a weighted fusion result;

[0030] Perform Sigmoid normalization on the weighted fusion result to obtain the dynamic matching degree, which is used to represent the matching probability between the candidate animal and the lost pet.

[0031] In one embodiment, predict the pet's activity range according to the dynamic matching degree and generate a visualized heat map, including:

[0032] According to the visual feature, semantic feature and spatio-temporal joint vector, obtain the basic movement speed parameter and calculate the body size adjustment factor;

[0033] Based on the behavior feature in the semantic feature, calculate the behavior activity intensity coefficient through natural language processing analysis;

[0034] Combine the basic movement speed parameter, body size adjustment factor, behavior activity intensity coefficient and lost time to calculate the dynamic activity radius;

[0035] Based on the dynamic activity radius, call the Gaode Map API to generate a dynamic gradient heat circle, with the lost position coordinate as the center and the dynamic activity radius as the radius;

[0036] Mark the positions in the dynamic gradient heat map with a dynamic matching degree higher than the preset threshold as red hot spots, and generate a visual heat map.

[0037] In one embodiment, the basic movement speed parameter is obtained through the following steps:

[0038] Construct a database containing 100,000 pet movement trajectories. Each pet movement trajectory is associated with visual feature tags, semantic behavior tags, and spatio-temporal environment data. The spatio-temporal environment data includes the timestamp of the trajectory point and the Gaussian kernel encoding of the corresponding position.

[0039] Train the spatio-temporal - behavior dual-channel LSTM network model through the database to obtain a pet activity prediction model.

[0040] Input the visual features, semantic features, and spatio-temporal joint vector into the pet activity prediction model to obtain spatio-temporal - behavior joint features.

[0041] Compare the body size with the preset standard body size matrix of the same breed, calculate the body size difference ratio, and adjust the spatio-temporal - behavior joint features based on the calculated body size difference ratio to generate a movement speed parameter after body size calibration.

[0042] Perform statistical analysis on the movement speed parameter after body size calibration, and take the median value of pets of the same breed to obtain the basic movement speed parameter.

[0043] In one embodiment, screen and sort the dynamic matching degree based on the visual heat map and the preset screening threshold, and output an interactive map interface, including:

[0044] Compare the dynamic matching degree with the preset screening threshold, screen out the candidate positions with a dynamic matching degree higher than the preset screening threshold, and sort them from high to low according to the dynamic matching degree.

[0045] Load the map base map through the Echarts engine or Mapbox GL JS framework, and set the initial center point of the map base map based on the lost position coordinates.

[0046] Overlay the visual heat map and the candidate positions on the map base map to generate an interactive map interface.

[0047] Among them, based on the interactive map interface, clickable marker points are embedded in the candidate positions. The clickable marker points are used to display a pet feature comparison chart after clicking. The pet feature comparison chart includes a visual feature comparison matrix and a semantic feature comparison matrix.

[0048] Integrate a time slider control in the interactive map interface. The time slider control is used to dynamically display the distribution of the dynamic matching degree at different time points.

[0049] Second aspect, this application also provides a pet searching geographical information matching system based on machine learning and natural language processing, including:

[0050] A data acquisition module, configured to acquire a pet image, a text description, and lost space-time information input by a user, where the lost space-time information includes lost position coordinates and a lost time;

[0051] A feature extraction module, configured to extract visual features of the pet image and semantic features of the text description through a deep learning model, where the visual features include breed, color, body type, and special markings, and the semantic features include breed entities, color modifiers, and behavioral features;

[0052] A pet searching matching module, configured to encode the lost space-time information into a space-time joint vector, fuse the space-time joint vector, visual features, and semantic features, and calculate a dynamic matching degree;

[0053] A pet searching visualization module, configured to predict a pet activity range according to the dynamic matching degree and generate a visualized heat map, and perform screening and sorting on the dynamic matching degree based on the visualized heat map and a preset screening threshold, and output an interactive map interface.

[0054] Third aspect, this application also provides a computer device, including a memory and a processor, where the memory stores a computer program, and the steps in the first aspect are implemented when the processor executes.

[0055] Fourth aspect, this application also provides a computer-readable storage medium, on which a computer program is stored, and the steps in the first aspect are implemented when the computer program is executed by a processor.

[0056] The above pet - seeking geographical information matching method based on machine learning and natural language processing can comprehensively collect key information related to the lost pet by obtaining the pet image, text description, and lost space - time information input by the user, providing a comprehensive data basis for subsequent feature extraction and matching. Secondly, visual features such as the breed, color, body shape, and special marks of the pet image and semantic features such as breed entities, color modifiers, and behavior features of the text description can be extracted through deep - learning models respectively. Compared with the traditional pet - seeking method that only relies on simple text descriptions, it can describe the pet's features more comprehensively and meticulously, reducing the matching error caused by incomplete information and improving the accuracy of feature expression. Subsequently, the lost space - time information is encoded into a space - time joint vector and fused with visual features and semantic features, adding constraints in the space - time dimension to the matching process, making the matching result more in line with the actual situation. Finally, the pet's activity range is predicted according to the dynamic matching degree and a visual heat map is generated, which can intuitively display the areas where the pet may appear, providing clear direction guidance for pet - seeking personnel. And by screening and sorting the dynamic matching degree based on the visual heat map and preset screening thresholds, an interactive map interface is output, facilitating users to query and screen information according to their own needs and improving the efficiency of pet - seeking.

[0057] Compared with traditional pet - seeking methods, this method is based on machine learning and natural language processing technologies, achieving automatic and precise matching and positioning of lost pet information. The comprehensive information acquisition and multi - dimensional feature extraction provide sufficient information for the matching process, enhancing the adaptability to different description methods and information sources during pet - seeking. The dynamic matching degree calculation that fuses space - time information and the visualization of the result output further enhances the positioning accuracy of the lost pet's location and the accuracy of information matching, improving the success rate of pet - seeking and reducing the time and cost of pet - seeking. In addition, the interactive map interface provides a reliable operation platform for pet - seeking personnel, effectively improving the efficiency and quality of pet - seeking and providing strong support for retrieving lost pets. Brief Description of the Drawings

[0058] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following will briefly introduce the drawings required for use in the description of the embodiments or related technologies. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0059] Figure 1 Flowchart of the pet - seeking geographical information matching method based on machine learning and natural language processing provided for an exemplary embodiment of the present invention;

[0060] Figure 2Schematic diagram of a pet - finding geographical information matching system based on machine learning and natural language processing provided for an exemplary embodiment of the present invention. Detailed implementation manners

[0061] In order to make the objectives, technical solutions and advantages of the present application clearer and more understandable, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0062] In one embodiment, as Figure 1 shown, a pet - finding geographical information matching method based on machine learning and natural language processing is provided. In this embodiment, this method is illustrated by taking its application to a terminal as an example. It can be understood that this method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is realized through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0063] S101: Obtain the pet image, text description and lost space - time information input by the user. The lost space - time information includes the lost position coordinates and the lost time.

[0064] Specifically, the pet - related information input by the user can be obtained through a user terminal such as a mobile phone APP or a web page. Among them, the pet image can include key information such as the pet's face and body characteristics, and the text description of the lost pet is obtained, such as information about the pet's habits, personality traits, and the state when it was lost. In addition, the lost space - time information of the lost pet needs to be obtained. The lost position coordinates can be obtained by means of the GPS positioning function of the mobile phone, or the user manually marks the specific location where the pet was lost on the map for obtaining, and the lost time can be accurate to the specific year, month, day, etc., for the convenience of subsequent pet - finding processing.

[0065] S102: Extract the visual features of the pet image and the semantic features of the text description through a deep - learning model. The visual features include breed, color, body type and special marks, and the semantic features include breed entities, color modifiers and behavioral features.

[0066] Specifically, for pet images, an extended Transformer model, such as ViT or the VisionTransformer deep learning framework, can be used to segment the images into multiple small segments before feeding them into the model for processing. This model can automatically learn and extract visual features such as the pet's breed, color, size, and special markings. For example, by training on a large number of pet images, the model can accurately identify a lost pet as a Chinese native dog with yellow and white fur, a medium build, and a black birthmark on its right hind leg. For text descriptions, pre-trained models based on the Transformer architecture, such as BERT or RoBERTa, can be used, combined with natural language processing techniques such as text preprocessing, part-of-speech tagging, and entity recognition, to extract semantic features such as breed entities, color modifiers, and behavioral characteristics. For example, in a text description such as "My white kitten is very lively and loves to chase a ball," the model can extract "kitten" as the breed entity, "white" as the color modifier, and "loves to chase a ball" as the behavioral feature.

[0067] S103: Encode the lost spatiotemporal information into a spatiotemporal joint vector, fuse the spatiotemporal joint vector, visual features, and semantic features, and calculate the dynamic matching degree.

[0068] Specifically, the coordinates of the lost location can be converted into a vector form through a specific encoding method, and the loss time can be encoded into a vector according to the relevant algorithm of the time series, and then the two can be combined into a spatiotemporal joint vector. Based on this spatiotemporal joint vector, visual features and semantic features, corresponding weights can be assigned according to different types of features for fusion. For example, for some pets with obvious special markings, the weight of the visual features of the special markings can be set higher. Subsequently, a preset matching algorithm such as cosine similarity calculation, machine learning model prediction, etc. can be used to calculate the dynamic matching degree based on the fused features. This dynamic matching degree can reflect the similarity between the currently acquired information and the information of other pets in the database.

[0069] S104: Predicting the pet's activity range based on the dynamic matching degree and generating a visual heat map, filtering and sorting the dynamic matching degree based on the visual heat map and a preset filtering threshold, and outputting an interactive map interface.

[0070] Specifically, based on the calculated dynamic matching degree, the possible activity range of the pet can be predicted by combining the pet's habits, the environment of the lost location, and time factors, using relevant algorithm models. And by calling tools such as the Gaode Map API and Echarts, a visual heat map is generated. In this visual heat map, the darker the color of the area, the higher the possibility of the pet appearing. In addition, by setting a preset screening threshold, the information with a dynamic matching degree higher than the threshold is screened, and sorted from high to low according to the dynamic matching degree, and an interactive map interface is output. Through this interactive map interface, users can not only intuitively see the matching results on the map, but also click on the marked points on the map to obtain detailed pet information, such as the time and location of finding the pet, the description of the pet's characteristics, etc., which is convenient for users to further confirm whether it is their lost pet, greatly improving the efficiency and accuracy of pet searching.

[0071] In the above method, by obtaining the pet image, text description, and lost spatio-temporal information, the visual, semantic, and spatio-temporal features can be comprehensively integrated, solving the problem of incomplete information caused by a single data type in traditional pet-searching methods. Secondly, by using a deep learning model to extract the visual features and semantic features of the corresponding lost pet, multi-dimensional capture of pet features is achieved, which not only improves the richness of the features, but also avoids the limitations of single-modal data in describing pet features. And by encoding the lost spatio-temporal information into a spatio-temporal joint vector and fusing it with the visual features and semantic features, the time and space rules of the pet's loss can be captured, making the calculated dynamic matching degree closer to the actual situation. Finally, predicting the pet's activity range based on the dynamic matching degree and generating a visual heat map, screening and sorting the dynamic matching degree based on a preset screening threshold, and outputting an interactive map interface can not only intuitively view the possible activity areas of the pet, but also quickly locate the high-probability pet-searching areas, reducing the time cost for users to search for information in a large amount of data and improving the accuracy and practicality of pet searching.

[0072] In an exemplary embodiment, the visual features of the pet image and the semantic features of the text description are extracted by a deep learning model. The visual features include breed, color, body type, and special marks, and the semantic features include breed entity, color modifier, and behavior features, including:

[0073] Using the Vision Transformer model, the pet image is segmented into multiple pixel blocks, and each pixel block is linearly projected to generate initial image features, and the initial image features are optimized based on a multi-level self-attention mechanism to generate optimized image features;

[0074] Perform multi-instance weak supervision learning on optimized image features, and use the MS-Captivator model to generate structured pet tags. The structured pet tags include breed classification codes, color spectrum vectors, body shape topologies, and special marker position matrices;

[0075] Perform multi-head self-attention dynamic weighting on the structured pet tags, calculate the weights of each feature channel through the following formula, and output visual features:

[0076] ;

[0077] where Q is the query vector, is the key vector of the i-th feature channel, and d is the feature dimension;

[0078] Input the text description into the RoBERTa model for semantic parsing to generate initial semantic features, and use the structured pet tags as cross-modal prior knowledge. Through the cross-modal cross-attention mechanism, fuse the visual features with the initial semantic features to generate enhanced semantic features;

[0079] Based on the enhanced semantic features, calculate the difference loss between the color spectrum vector and the color description in the text description through the color semantic calibration network, and backpropagate to optimize the parameters of the MS-Captivator model and the RoBERTa model, and output semantic features strongly associated with the visual features.

[0080] Specifically, when processing images, the Vision Transformer model is different from traditional convolutional neural networks that extract local features by sliding convolutions based on local receptive fields. Instead, it divides the image into multiple non-overlapping pixel blocks of a fixed size for processing. Secondly, linear projection operations can be performed on each segmented pixel block to convert it into a low-dimensional feature vector. This initial image feature contains the local feature information of each pixel block. Based on this initial image feature, through a multi-level self-attention mechanism, when processing each pixel block, the information of other pixel blocks in the image is considered, thereby capturing long-range dependencies in the image. For example, when processing an image of a pet cat, the model not only focuses on the pixel blocks of the face but can also associate with the pixel block information of the tail, limbs, etc. through the self-attention mechanism. After processing each pixel block and comprehensively considering the global information, an optimized image feature is generated. Subsequently, the obtained optimized image feature is subjected to multi-instance weak supervised learning. Multi-instance weak supervised learning means that in the case where the training data annotation is not very accurate, learning can be carried out by combining multiple related instances into a bag. In this embodiment, the MS-Captivator model is used to process the optimized image feature to generate a structured pet label. Among them, the breed classification code digitizes the breed information of the pet for subsequent model processing and recognition; the color spectrum vector models in the Lab or HSV color space to quantify the distribution, hue, and saturation information of the main colors in the pet image, enhancing the discriminability of color semantics, thereby accurately describing features such as the hue and saturation of the pet's color; the body shape topology map describes information such as the body shape outline and proportion of the pet through certain algorithms and data structures; the special marker position matrix records the specific position information of special markers such as birthmarks and spots on the pet in the image.

[0081] Specifically, the multi-head self-attention mechanism is an extension of the self-attention mechanism. It can calculate in parallel through multiple different attention heads, capture the relationships between features from different perspectives, and can extract feature information more comprehensively. Therefore, the above formula can be used to dynamically adjust the importance of different feature channels such as breed classification encoding, color spectrum vector, body shape topology map, and special marker position matrix in the generation of the final visual feature by the multi-head self-attention mechanism, and then output the weighted visual feature to highlight more critical pet features. And the text description input by the user is input into the RoBERTa model. Among them, RoBERTa is a pre-trained language model based on the Transformer architecture. By pre-training on a large amount of text data, it can learn rich language knowledge and semantic representation capabilities. The model performs semantic parsing on the text description, including operations such as word segmentation, part-of-speech tagging, and syntactic analysis, to generate initial semantic features. Since images and texts belong to different modalities of information, the structured pet label, as a structured representation of image features, can provide additional image-related knowledge for the generation of text semantic features. Therefore, the above-generated structured pet label can be used as cross-modal prior knowledge. Through the cross-modal cross-attention mechanism, the model can dynamically allocate weights for fusion according to the degree of association between visual features and initial semantic features, generating enhanced semantic features to make the semantic information more comprehensive and accurate. In addition, a color semantic calibration network can be introduced. By comparing the color spectrum vector in the structured pet label and the color description content in the text description, the difference between the two is calculated and converted into a loss value. Subsequently, through the backpropagation algorithm, this loss value is fed back to the MS-Captivator model and the RoBERTa model, enabling the subsequent models to more accurately associate visual features and semantic features when processing pet images and text descriptions. Finally, semantic features strongly associated with visual features are output, ensuring that the features extracted from images and texts can complement each other, thereby improving the accuracy and reliability of the entire feature extraction process and providing high-quality feature data for subsequent tasks such as lost pet geographical information matching.

[0082] In an exemplary embodiment, encoding the lost spatio-temporal information as a spatio-temporal joint vector includes:

[0083] Taking the lost position coordinates as the center point, calculating the Gaussian kernel encoding of geographical coordinates through the following formula:

[0084] ;

[0085] where σ = 0.1 is the sensitivity coefficient, obtained by training with historical data, is the lost position coordinates;

[0086] Calculating the decay factor according to the lost time through the following calculation formula:

[0087] ;

[0088] t is the loss time, and 48 is the reference decay period;

[0089] Concatenate the Gaussian kernel encoding and the decay factor to generate a spatio-temporal feature vector, and normalize the spatio-temporal feature vector to obtain a spatio-temporal joint vector.

[0090] Specifically, the Gaussian kernel function can map geographical coordinates to a continuous feature space. Therefore, with the lost position coordinates as the center point, through Gaussian kernel encoding, the similarity between geographical coordinates can be captured, making the similar geographical coordinates have a higher similarity in the feature space. The sensitivity coefficient σ in the above formula is used to control the width of the Gaussian kernel function and determines the similarity threshold between geographical coordinates. A smaller σ value indicates a higher sensitivity to geographical coordinates, while a larger σ value indicates a lower sensitivity. And the time decay factor can be calculated according to the loss time to reflect the impact of time on the pet's activity range. Among them, the time decay factor reflects the impact of the loss time on the pet's activity range. As time goes by, the pet's activity range will gradually expand, so the matching weight will gradually decrease. Through this time decay factor, the weight of the matching result can be dynamically adjusted to make the matching result more in line with the actual situation. The reference decay period is an empirical value, indicating the reference period of time decay. In this embodiment, the reference decay period is set to 48 hours, indicating that the impact of time decay is more significant within 48 hours. Subsequently, the Gaussian kernel encoding and the time decay factor can be concatenated to generate a spatio-temporal feature vector, and the spatio-temporal feature vector is normalized to ensure the rationality and accuracy of the spatio-temporal feature vector, thereby improving the accuracy and efficiency of pet searching, and finally obtaining a spatio-temporal joint vector.

[0091] In an exemplary embodiment, fuse the visual feature, semantic feature and spatio-temporal joint vector, and calculate the dynamic matching degree, including:

[0092] Obtain a candidate animal image, and extract features from the candidate animal image through a deep learning model to obtain the visual feature of the candidate animal image;

[0093] Calculate the cosine similarity between the visual feature and the visual feature of the candidate animal image;

[0094] Calculate the semantic similarity between the semantic feature and the text description of the candidate animal through the BERTScore algorithm;

[0095] Perform weighted fusion on the spatio-temporal joint vector, cosine similarity and semantic similarity to obtain a weighted fusion result;

[0096] Perform Sigmoid normalization on the weighted fusion result to obtain the dynamic matching degree, which is used to represent the matching probability between the candidate animal and the lost pet.

[0097] Specifically, the cosine similarity measures the similarity between two vectors by calculating the cosine value of the angle between them. The cosine similarity between the visual features of the lost pet and the visual features of the candidate animal image can be calculated according to the cosine similarity formula. If the similarity value is closer to 1, it indicates that the directions of the two visual feature vectors are more similar, that is, the similarity degree between the lost pet and the candidate animal in appearance is higher. The BERTScore algorithm is built based on the pre-trained BERT model (Bidirectional Encoder Representations from Transformers, BERT). The BERT model can be pre-trained on large-scale text data to learn rich language semantic representations. That is, the semantic features of the lost pet and the text description of the candidate animal can be input into the preset BERT model respectively, and then the semantics of the words, the grammar structure of the sentences, and the context of the text can be analyzed. Subsequently, the semantic similarity can be calculated by comparing the distances of the two texts in the semantic representation space output by the model through the BERTScore algorithm. This semantic similarity value reflects the proximity of the semantics of the two, and the higher the value, the more similar the semantics. Since visual features, semantic features, and spatio-temporal information all play important roles in measuring the matching degree between the candidate animal and the lost pet, but the importance of the three may be different, they can be fused by weighted fusion according to the preset weights to obtain the weighted fusion result. And in order to convert it into a numerical value that can intuitively represent the matching probability between the candidate animal and the lost pet, the Sigmoid function can be used for normalization to output the dynamic matching degree. This dynamic matching degree represents the matching probability between the candidate animal and the lost pet.

[0098] In an exemplary embodiment, predicting the pet's activity range based on the dynamic matching degree and generating a visual heat map includes:

[0099] Obtain the basic movement speed parameter according to the visual features, semantic features, and spatio-temporal joint vector, and calculate the body size adjustment factor;

[0100] Based on the behavior features in the semantic features, calculate the behavior activity intensity coefficient through natural language processing analysis;

[0101] Combine the basic movement speed parameter, the body size adjustment factor, the behavior activity intensity coefficient, and the lost time to calculate the dynamic activity radius;

[0102] Based on the dynamic activity radius, call the Gaode Map API to generate a dynamic gradient heat circle. The dynamic gradient heat circle takes the lost location coordinates as the center and the dynamic activity radius as the radius;

[0103] Mark the locations in the dynamic gradient heat circle with a dynamic matching degree higher than the preset threshold as red hot spots to generate a visualized heat map.

[0104] Specifically, the basic movement speed parameters of different pet breeds can be preset. For example, the basic movement speed of small dogs is 2 kilometers per hour, medium-sized dogs is 3 kilometers per hour, and large dogs is 4 kilometers per hour. These basic movement speed parameters reflect the average movement speed of different breeds of pets and provide basic data for subsequent prediction of the activity range. And according to the standard body size information of the same breed of pets and the body size information in the visual features, the body size adjustment factor can be calculated to adjust the basic movement speed to make the prediction result more in line with the actual situation. In addition, a pre-trained model based on Transformer can be used to perform dependency syntactic analysis on the behavioral features in the text description, identify the subject-predicate relationship, verb-object relationship, etc. in the text, extract the semantic information of the behavioral features, and calculate the behavioral activity intensity coefficient. This behavioral activity intensity coefficient reflects the behavioral activity level of the pet. Pets with higher activity levels usually have a larger activity range, so the behavioral activity intensity coefficient can be increased accordingly. Finally, the basic movement speed parameter, body size adjustment factor, behavioral activity intensity coefficient, and lost time can be combined to calculate the dynamic activity radius. This dynamic activity radius reflects the possible activity range of the pet during the lost time, and the activity radius will gradually expand over time and with different behavioral characteristics of the pet.

[0105] Furthermore, call the Gaode Map API to generate a dynamic gradient heat circle with the lost location coordinates as the center and the dynamic activity radius as the radius. This dynamic gradient heat circle can reflect the probability of the pet's appearance through the depth of color, and the darker the color, the higher the probability. And the locations in the dynamic gradient heat circle with a dynamic matching degree higher than the preset threshold can be marked as red hot spots. This red hot spot area can represent the area where the probability of the pet's appearance is high and the possibility of matching the lost pet is large after considering the visual features, semantic features, spatio-temporal information, and movement-related factors of the lost pet. Finally, integrating the marked red hot spot area with the dynamic gradient heat circle can obtain a visualized heat map. By viewing this map, users can intuitively understand the high-probability areas where the lost pet may appear, further improving the efficiency and accuracy of pet searching.

[0106] In an exemplary embodiment, the basic movement speed parameter is obtained through the following steps:

[0107] Construct a database containing 100,000 pet movement trajectories, where each pet movement trajectory is associated with visual feature tags, semantic behavior tags, and spatio-temporal environmental data. The spatio-temporal environmental data includes the timestamp of the trajectory point and the Gaussian kernel encoding of the corresponding location.

[0108] Train the spatio-temporal - behavior dual-channel LSTM network model through the database to obtain a pet activity prediction model.

[0109] Input the visual features, semantic features, and spatio-temporal joint vector into the pet activity prediction model to obtain spatio-temporal - behavior joint features.

[0110] Compare the body size with the preset standard body size matrix of the same breed, calculate the body size difference ratio, and adjust the spatio-temporal - behavior joint features based on the calculated body size difference ratio to generate a movement speed parameter with body size calibration.

[0111] Conduct statistical analysis on the movement speed parameter with body size calibration, and take the median value of pets of the same breed to obtain the basic movement speed parameter.

[0112] Specifically, the visual feature tags in the pet movement trajectory can record in detail the appearance information of the pet, such as breed, color, body size, special marks, etc. The semantic behavior tags can describe the semantic information such as the pet's behavior habits and personality characteristics, which helps to understand the pet's behavior habits. The spatio-temporal environmental data can effectively represent the geographical location of the pet and the information of its surrounding environment. Subsequently, the constructed database can be used to train the spatio-temporal - behavior dual-channel LSTM (Long Short-Term Memory) network model. Among them, the spatio-temporal - behavior dual-channel LSTM network model has two channels. One channel focuses on processing spatio-temporal environmental data and analyzing the changing rules of the pet movement trajectory over time and space; the other channel focuses on processing semantic behavior tags and visual feature tags to explore the associations between the visual features, behavior habits, and movement patterns of different pets. By continuously inputting a large amount of pet movement trajectory data into this model for training, the model can learn the activity rules of pets under different visual features, semantic behaviors, and spatio-temporal environmental conditions, and finally obtain a pet activity prediction model that can accurately predict the activities of pets. Input the visual features, semantic features, and spatio-temporal joint vector of the currently lost pet into the trained pet activity prediction model. This model can deeply analyze and fuse these input information, and extract spatio-temporal - behavior joint features that can comprehensively reflect the spatio-temporal activity characteristics and behavior patterns of the pet. This joint feature contains information such as the movement trend that the pet may generate based on its appearance and behavior habits under specific spatio-temporal conditions.

[0113] Further, compare the size of the lost pet with a preset standard size matrix of the same breed. This preset standard size matrix of the same breed is obtained through statistical analysis of the size data of a large number of pets of the same breed and has certain representativeness and standardness. Through comparison, the difference ratio between the current pet's size and the standard size can be calculated. Since size differences can affect the movement speed of pets, for example, pets with a relatively large size and deviating from the standard size may have a relatively slow movement speed. Therefore, the spatio-temporal and behavior joint features can be adjusted based on this size difference ratio to more accurately reflect the influence of the pet's size factor on the movement speed, and then generate the movement speed parameters after size calibration. And for pets of the same breed, the median of their movement speed parameters can be calculated. Among them, the median is the value located in the middle position after sorting a set of data from small to large. Compared with the average value, it is more resistant to the interference of outliers and has better robustness. Therefore, the basic movement speed parameter can be obtained by taking the median value of the movement speed parameters of pets of the same breed. This parameter serves as a basic reference value for subsequent calculation of pet movement-related indicators and provides a reliable basis for accurately predicting the pet's activity range.

[0114] In an exemplary embodiment, filter and sort the dynamic matching degrees based on the visual heat map and a preset filtering threshold, and output an interactive map interface, including:

[0115] Compare the dynamic matching degree with the preset filtering threshold, filter out the candidate positions with a dynamic matching degree higher than the preset filtering threshold, and sort them from high to low according to the dynamic matching degree;

[0116] Load the map base map through the Echarts engine or the Mapbox GL JS framework, and set the initial center point of the map base map based on the lost position coordinates;

[0117] Overlay the visual heat map and the candidate positions on the map base map to generate an interactive map interface;

[0118] Among them, based on the interactive map interface, clickable marker points are embedded in the candidate positions. The clickable marker points are used to display a pet feature comparison map after clicking. The pet feature comparison map includes a visual feature comparison matrix and a semantic feature comparison matrix;

[0119] Integrate a time slider control in the interactive map interface. The time slider control is used to dynamically display the distribution of dynamic matching degrees at different time points.

[0120] Specifically, the calculated dynamic matching degree can be compared with a preset screening threshold, such as 0.8, to screen out candidate positions with a dynamic matching degree higher than the preset screening threshold. Then, the screened candidate positions are sorted in descending order according to the dynamic matching degree, ensuring that the user can view the position with the highest matching degree first, thereby improving the efficiency of pet searching. Subsequently, the map base map can be loaded through the Echarts engine or the Mapbox GL JS framework. Among them, the Echarts engine can provide rich data visualization functions, while the Mapbox GL JS framework can provide efficient rendering and interaction functions for geographical data. Then, the initial center point of the map base map can be set according to the lost position coordinates, ensuring that the initial display area of the map is related to the pet lost position. By overlaying the generated visual heat map on the map base map, an interactive map interface can be generated. Moreover, in the interactive map interface, clickable marker points can be embedded in the candidate positions. After the user clicks on the marker point, a pet feature comparison chart can be displayed. The pet feature comparison chart includes a visual feature comparison matrix and a semantic feature comparison matrix. Among them, the visual feature comparison matrix can display the comparison of the pet's breed, color, body type, and special marks; the semantic feature comparison matrix can display the comparison of the pet's breed entity, color modifier, and behavior characteristics, which helps the user to compare the characteristics of the lost pet and the found pet in detail. In addition, the interactive map interface can integrate a time slider control to dynamically display the distribution of the dynamic matching degree at different time points. Schematically, the time slider control allows the user to select a specific time point and update the heat map and candidate positions according to the selected time point, dynamically displaying the matching degree score, further assisting in finding the lost pet.

[0121] Based on the same inventive concept, as Figure 2 shown, the embodiment of the present application also provides a pet searching geographical information matching system 200 based on machine learning and natural language processing. The implementation solution provided by this system to solve problems is similar to the implementation solution described in the above method. Therefore, the specific limitations in one or more system embodiments of the pet searching geographical information matching system based on machine learning and natural language processing can be referred to the limitations of the pet searching geographical information matching method based on machine learning and natural language processing in the above text, and will not be elaborated here. The system includes:

[0122] A data acquisition module 201, configured to acquire the pet image, text description, and lost spatio-temporal information input by the user, where the lost spatio-temporal information includes lost position coordinates and lost time;

[0123] A feature extraction module 202, configured to extract the visual features of the pet image and the semantic features of the text description through a deep learning model. The visual features include breed, color, body type, and special marks, and the semantic features include breed entity, color modifier, and behavior characteristics;

[0124] The pet search matching module 203 is used to encode the lost spatio-temporal information into a spatio-temporal joint vector, fuse the spatio-temporal joint vector, visual features, and semantic features, and calculate the dynamic matching degree;

[0125] The pet search visualization module 204 is used to predict the pet's activity range according to the dynamic matching degree and generate a visualized heat map, and filter and sort the dynamic matching degree based on the visualized heat map and a preset filtering threshold, and output an interactive map interface.

[0126] In the above system, the data acquisition module 201 can acquire the pet image, text description, and lost spatio-temporal information input by the user, effectively solving the problems of incomplete and inaccurate information acquisition in traditional pet search methods, comprehensively integrating the key information related to the lost pet, and thus providing a comprehensive data basis for subsequent pet search analysis. The feature extraction module 202 can extract the visual features of the pet image and the semantic features of the text description through a deep learning model, and can characterize the features of the pet from multiple dimensions, avoiding the limitation that traditional methods are difficult to accurately identify the pet only relying on simple descriptions. The pet search matching module 203 can encode the lost spatio-temporal information into a spatio-temporal joint vector, and fuse visual features and semantic features through the lost time and location information to calculate the dynamic matching degree, which not only effectively solves the problem that traditional pet search matching ignores spatio-temporal factors, but also can more scientifically evaluate the matching possibility, making the matching result more in line with the actual situation and improving the accuracy and reliability of the matching. The pet search visualization module 204 can predict the pet's activity range according to the dynamic matching degree to generate a visualized heat map, and filter and sort the dynamic matching degree through a preset filtering threshold, and output an interactive map interface. This module not only effectively solves the problems of unintuitive display and poor interactivity of traditional pet search results, but also can intuitively present the areas where the pet may appear, helping users to query and filter according to their own needs, and improving the efficiency of pet search.

[0127] In an exemplary embodiment, the present invention also provides a computer device, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the steps of the pet search geographical information matching method based on machine learning and natural language processing of the present application. A multi-core processor is preferably used to improve the parallel processing ability of the system. Memory: Provide sufficient temporary storage space to support the operation of the program and the processing of data. The memory capacity should be large enough to accommodate a large amount of supply information and computing tasks.

[0128] In an exemplary embodiment, the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the pet search geographical information matching method based on machine learning and natural language processing of the present application are implemented. The computer-readable storage medium may include: read-only memory (ROM, ReadOnly Memory), random access memory (RAM, Random Access Memory), solid state drives (SSD, SolidState Drives), optical discs, etc. Among them, the random access memory may include resistive random access memory (ReRAM, Resistance Random Access Memory) and dynamic random access memory (DRAM, Dynamic RandomAccess Memory).

[0129] The above-described embodiments merely represent several implementation manners of the embodiments of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the embodiments of the application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the embodiments of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the embodiments of the present application.

Claims

1. A pet-searching geographic information matching method based on machine learning and natural language processing, characterized in that: The method comprises: Obtaining a pet image, text description, and lost space-time information input by a user, wherein the lost space-time information includes lost location coordinates and lost time; Extract visual features of the pet image using a Vision Transformer model, and extract semantic features of the text description using a RoBERTa model. The visual features include breed, color, size, and markings, and the semantic features include breed entities, color modifiers, and behavioral features. Encoding the lost spatiotemporal information into a spatiotemporal joint vector, fusing the spatiotemporal joint vector, the visual features, and the semantic features to calculate a dynamic matching degree; Predicting the pet's activity range based on the dynamic matching degree and generating a visual heat map, filtering and sorting the dynamic matching degree based on the visual heat map and a preset filtering threshold, and outputting an interactive map interface; The step of encoding the lost spatiotemporal information into a spatiotemporal joint vector includes: Taking the lost location coordinates as the center point, the Gaussian kernel encoding of the geographic coordinates is calculated using the following formula: ； Where σ=0.1 is the sensitivity coefficient, which is obtained through historical data training. is the lost location coordinates; The attenuation factor is calculated according to the loss time using the following formula: ； t is the loss time, 48 is the reference decay period, in hours; The Gaussian kernel code and the attenuation factor are concatenated to generate a spatiotemporal feature vector, and the spatiotemporal feature vector is normalized to obtain the spatiotemporal joint vector.

2. The method according to claim 1, characterized in that The extracting of visual features of the pet image by the Vision Transformer model and the extracting of semantic features of the text description by the RoBERTa model include: Segmenting the pet image into a plurality of pixel blocks, and inputting each pixel block into the Vision Transformer model for processing, and generating initial image features through a multi-level self-attention mechanism; Performing multi-instance weakly supervised learning on the initial image features, and generating a structured pet label using the MS-Captivator model, wherein the structured pet label includes a breed classification code, a color pedigree vector, a body shape topology map, and a marker position matrix; The structured pet tag is dynamically weighted by multi-head self-attention, and the weight of each feature channel is calculated using the following formula to output the visual feature: ； Where Q is the query vector, is the key vector of the i-th feature channel, and d is the feature dimension; Input the text description into the RoBERTa model for semantic parsing to generate initial semantic features, and use the structured pet tags as cross-modal prior knowledge to fuse the visual features with the initial semantic features through an attention mechanism to generate enhanced semantic features; Based on the enhanced semantic features, the difference loss between the color spectrum vector and the color description in the text description is calculated through a color semantic calibration network, and the parameters of the MS-Captivator model and the RoBERTa model are optimized through backpropagation to output the semantic features that are strongly associated with the visual features.

3. The method according to claim 1, characterized in that The fusing of the visual features, the semantic features, and the spatiotemporal joint vector to calculate the dynamic matching degree includes: Calculating the cosine similarity between the visual feature and the visual feature of the candidate animal image; Calculate the semantic similarity between the semantic feature and the text description of the candidate animal using the BERTScore algorithm; Performing weighted fusion on the spatiotemporal joint vector, the cosine similarity, and the semantic similarity to obtain a weighted fusion result; The weighted fusion result is subjected to Sigmoid normalization to obtain the dynamic matching degree, which is used to represent the matching probability between the candidate animal and the lost pet.

4. The method according to claim 1, wherein The method of predicting the pet's activity range based on the dynamic matching degree and generating a visual heat map includes: Loading basic movement speed parameters and calculating a body shape adjustment factor according to the visual features, the semantic features, and the spatiotemporal joint vector; Based on the behavioral features in the semantic features, calculating the behavioral activity intensity coefficient through dependency syntax analysis; Calculating a dynamic activity radius by combining the basic movement speed parameter, the body size adjustment factor, the behavioral activity intensity coefficient, and the loss time; Based on the dynamic activity radius, calling the Amap API to generate a dynamic gradient thermal circle, wherein the dynamic gradient thermal circle takes the lost location coordinates as the center and the dynamic activity radius as the radius; The locations in the dynamic gradient heat circle where the dynamic matching degree is higher than a preset threshold are marked as red hot spots to generate the visual heat map.

5. The method according to claim 4, characterized in that The basic moving speed parameter is obtained by the following steps: Constructing a database of 100,000 pet movement trajectories, each of which is associated with a visual feature label, a semantic behavior label, and spatiotemporal environmental data, including the timestamp of the trajectory point and the Gaussian kernel encoding of the corresponding position; The spatiotemporal-behavioral dual-channel LSTM network model is trained using the database to obtain a pet activity prediction model; Inputting the visual features, the semantic features, and the spatiotemporal joint vector into the pet activity prediction model to obtain a spatiotemporal-behavior joint feature; Comparing the body shape with a preset standard body shape matrix of the same breed, calculating the body shape difference ratio, and adjusting the spatiotemporal-behavioral joint feature based on the calculated body shape difference ratio to generate a body shape-calibrated movement speed parameter; Statistical analysis is performed on the movement speed parameters after body shape calibration, and the median value of pets of the same breed is taken to obtain the basic movement speed parameter.

6. The method according to claim 1, wherein The filtering and sorting of the dynamic matching degrees based on the visualized heat map and a preset filtering threshold, and outputting an interactive map interface, includes: Comparing the dynamic matching degree with the preset screening threshold, screening out candidate positions whose dynamic matching degree is higher than the preset screening threshold, and sorting the candidate positions from high to low according to the dynamic matching degree; Loading a map basemap through the Echarts engine or the Mapbox GL JS framework, and setting an initial center point of the map basemap based on the coordinates of the lost location; Overlaying the visualized heat map and the candidate locations onto the map base map to generate the interactive map interface; Wherein, based on the interactive map interface, clickable markers are embedded in the candidate locations, and the clickable markers are used to display a pet feature comparison chart after being clicked, and the pet feature comparison chart includes a visual feature comparison matrix and a semantic feature comparison matrix; A time slider control is integrated into the interactive map interface, and the time slider control is used to dynamically display the dynamic matching degree distribution at different time points.

7. A pet-finding geographic information matching system based on machine learning and natural language processing, characterized by: The system comprises: A data acquisition module is used to acquire a pet image, a text description, and lost time and space information input by a user, wherein the lost time and space information includes the lost location coordinates and the lost time; a feature extraction module, configured to extract visual features of the pet image using a Vision Transformer model and semantic features of the text description using a RoBERTa model, wherein the visual features include breed, color, size, and markings, and the semantic features include breed entities, color modifiers, and behavioral characteristics; A pet-finding matching module is configured to encode the lost spatiotemporal information into a spatiotemporal joint vector, fuse the spatiotemporal joint vector, the visual features, and the semantic features, and calculate a dynamic matching degree; A pet search visualization module is used to predict the pet's activity range based on the dynamic matching degree and generate a visual heat map, and to filter and sort the dynamic matching degree based on the visual heat map and a preset filtering threshold, and output an interactive map interface; The pet-searching and matching module further includes an encoding subunit, which is used to: Taking the lost location coordinates as the center point, the Gaussian kernel encoding of the geographic coordinates is calculated using the following formula: ； Where σ=0.1 is the sensitivity coefficient, which is obtained through historical data training. is the lost location coordinates; The attenuation factor is calculated according to the loss time using the following formula: ； t is the loss time, 48 is the reference decay period, in hours; The Gaussian kernel code and the attenuation factor are concatenated to generate a spatiotemporal feature vector, and the spatiotemporal feature vector is normalized to obtain the spatiotemporal joint vector.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images