A sports player career profile data graph construction method and system

Abstract

The application provides a sports athlete career archive data graph construction method and system, and relates to the technical field of data processing. The method comprises the following steps: based on a cross-modal correction coefficient, inputting image modal data and text modal data into a pre-trained single-double flow interaction model, performing semantic similarity alignment, and utilizing the cross-modal correction coefficient to optimize the alignment process to obtain a cross-modal entity set; according to the cross-modal entity set, combining a pre-defined domain rule and a deep pre-training language model to construct a relationship set between entities; and according to the cross-modal entity set and the relationship set between entities, storing entities as nodes and relationships as edges into a graph database to construct a sports athlete career archive knowledge graph. The application improves the completeness, accuracy and semantic consistency of athlete career archive knowledge extraction.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method and system for constructing a career archive data map of a sports athlete. Background Technology

[0002] In constructing a basketball player's career profile, it is necessary to integrate various forms of information related to the player's games to accurately record their on-court performance and technical characteristics, providing a reliable basis for subsequent profile organization and on-court performance analysis. Image modal data primarily consists of high-definition footage of key technical movements captured during games. Specifically, this includes close-up screenshots of wrist rotation angles and fingertip power generation during three-point shots, dynamic frame screenshots of body weight shifts and foot landing positions during breakthroughs, and scene screenshots of arm extension angles, positioning, and distance from offensive players during defense. Text modal data includes detailed technical statistics for corresponding games (such as shooting percentage, breakthrough success rate, and defensive steals), and coach-specific comments on specific movements (e.g., insufficient wrist power during three-point shots resulting in a low shooting arc, or a fast first step during a breakthrough but with insufficient footwork). Effective association between the two is crucial to ensuring the accuracy and completeness of career profile information. Currently, when associating basketball player images with text data, conventional methods often involve first extracting the visual features of the image and the semantic features of the text separately, and then aligning the features using preset matching rules. However, in practical applications, it has been found that semantic correspondence discrepancies easily occur between the detailed visual features in basketball images (such as the specific angle of wrist rotation during a three-point shot, the distance between the foot landings during a breakthrough, and the range of arm extension during defense) and the concise descriptions of the action in the text (such as a low arc in a three-point shot, a lack of smooth footwork during a breakthrough, and precise defensive positioning). This may lead to insufficient alignment accuracy, thereby affecting the accurate identification of cross-modal entities in the athlete's career profile (such as the three-point shooting action and the commentary on a low arc), and may make it difficult to stably support the subsequent construction of a career profile knowledge graph. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide a method and system for constructing a data map of athletes' career records, which improves the completeness, accuracy and semantic consistency of knowledge extraction from athletes' career records.

[0004] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows: Firstly, a method for constructing a career archive data map of a sports athlete, the method comprising: Step 1: Obtain athlete career profile data containing image modal data and text modal data; convert the image modal data into visual semantic particle clusters; construct a feature projection manifold based on the visual semantic particle clusters; and delineate semantic sampling constraint polygons on the feature projection manifold. Step 2: Calculate the shape complexity index of the semantic sampling constraint polygon. The shape complexity index is quantified by analyzing the curvature changes of the polygon boundary points and the ratio of area to perimeter. Based on the spatial distribution characteristics of the visual semantic particle clusters, the semantic sampling constraint polygon is subdivided according to the shape complexity index to generate multiple visual feature partitions. The visual semantic particle clusters are mapped to the corresponding visual feature partitions, and the local feature entropy of the visual feature partitions is calculated. The local feature entropy and the shape complexity index are fused to generate cross-modal correction coefficients. Step 3: Based on the cross-modal correction coefficient, input the image modal data and text modal data into the pre-trained single- and dual-stream interaction model, perform semantic similarity alignment, and optimize the alignment process using the cross-modal correction coefficient to obtain a cross-modal entity set; Step 4: Based on the cross-modal entity set, and combining predefined domain rules with a deep pre-trained language model, construct a set of relationships between entities; Step 5: Based on the cross-modal entity set and the set of relationships between entities, store the entities as nodes and the relationships as edges in the graph database to construct a knowledge graph of athletes' career profiles.

[0005] Secondly, a system for constructing a career archive data map of sports athletes includes: The acquisition module is used to acquire sports athlete career profile data containing image modal data and text modal data, convert the image modal data into visual semantic particle clusters, construct a feature projection manifold based on the visual semantic particle clusters, and delineate semantic sampling constraint polygons on the feature projection manifold. The computation module is used to calculate the shape complexity index of the semantic sampling constraint polygon. The shape complexity index is quantified by analyzing the curvature changes of the polygon boundary points and the ratio of area to perimeter. Based on the spatial distribution characteristics of visual semantic particle clusters, the semantic sampling constraint polygon is subdivided according to the shape complexity index to generate multiple visual feature partitions. The visual semantic particle clusters are mapped to the corresponding visual feature partitions, and the local feature entropy of the visual feature partitions is calculated. The local feature entropy and the shape complexity index are fused to generate cross-modal correction coefficients. The alignment module is used to input image modal data and text modal data into a pre-trained single- and dual-stream interaction model based on cross-modal correction coefficients, perform semantic similarity alignment, and optimize the alignment process using cross-modal correction coefficients to obtain a cross-modal entity set. The building module is used to construct a set of relationships between entities based on a cross-modal entity set, combined with predefined domain rules and a deep pre-trained language model; The output module is used to store entities as nodes and relationships as edges in a graph database to construct a knowledge graph of athletes' career profiles, based on the cross-modal entity set and the set of relationships between entities.

[0006] Thirdly, a computing device, comprising: One or more processors; A storage device for storing one or more programs that, when executed by one or more processors, cause the one or more processors to implement the method.

[0007] Fourthly, a computer-readable storage medium storing a program that, when executed by a processor, implements the method.

[0008] The above-described solution of the present invention has at least the following beneficial effects: This invention transforms image modal data into visual semantic particle clusters and constructs feature projection manifolds and semantic sampling constraint polygons. This enables refined characterization of fine-grained visual features such as shooting wrist angles, dribbling footwork, and defensive postures, achieving structured decomposition of local image details from the source. Simultaneously, by combining shape complexity indices and local feature entropy to generate cross-modal correction coefficients, it adaptively quantifies the distribution complexity and dispersion of image visual features, effectively solving the problem of mismatch and inaccurate matching between fine-grained visual features and condensed text commentary semantics, improving the recognition accuracy of cross-modal entities such as three-point actions, dribbling postures, and defensive behaviors. The semantic sampling constraint polygons are recursively and adaptively subdivided until the particle distribution uniformity of the sub-regions meets a preset threshold, avoiding feature redundancy or loss caused by fixed subdivisions. By calculating the local feature entropy of each visual feature partition, the feature richness and discriminativeness of local regions can be objectively quantified, making the subsequent cross-modal alignment process more focused on high-information visual regions and improving the efficiency of multimodal data processing. Attached Figure Description

[0009] Figure 1 This is a schematic diagram of a method for constructing a career archive data map of a sports athlete, provided by an embodiment of the present invention.

[0010] Figure 2 This is a schematic diagram of a sports athlete career profile data mapping system provided by an embodiment of the present invention.

[0011] Figure 3 It is a spatial distribution map of visual semantic particle clusters.

[0012] Figure 4 This is a trend chart of cross-modal correction coefficient calculation.

[0013] Figure 5 It is a histogram of the semantic similarity distribution between images and text. Detailed Implementation

[0014] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0015] like Figure 1 As shown, an embodiment of the present invention proposes a method for constructing a career archive data map of a sports athlete, the method comprising the following steps: Step 1: Obtain athlete career profile data containing image modal data and text modal data; convert the image modal data into visual semantic particle clusters; construct a feature projection manifold based on the visual semantic particle clusters; and delineate semantic sampling constraint polygons on the feature projection manifold. Step 2: Calculate the shape complexity index of the semantic sampling constraint polygon. The shape complexity index is quantified by analyzing the curvature changes of the polygon boundary points and the ratio of area to perimeter. Based on the spatial distribution characteristics of the visual semantic particle clusters, the semantic sampling constraint polygon is subdivided according to the shape complexity index to generate multiple visual feature partitions. The visual semantic particle clusters are mapped to the corresponding visual feature partitions, and the local feature entropy of the visual feature partitions is calculated. The local feature entropy and the shape complexity index are fused to generate cross-modal correction coefficients. Step 3: Based on the cross-modal correction coefficient, input the image modal data and text modal data into the pre-trained single- and dual-stream interaction model, perform semantic similarity alignment, and optimize the alignment process using the cross-modal correction coefficient to obtain a cross-modal entity set; Step 4: Based on the cross-modal entity set, and combining predefined domain rules with a deep pre-trained language model, construct a set of relationships between entities; Step 5: Based on the cross-modal entity set and the set of relationships between entities, store the entities as nodes and the relationships as edges in the graph database to construct a knowledge graph of athletes' career profiles.

[0016] In this embodiment of the invention, image modal data is transformed into visual semantic particle clusters, and feature projection manifolds and semantic sampling constraint polygons are constructed. This enables refined characterization of fine-grained visual features such as shooting wrist angles, breakthrough footwork, and defensive postures, achieving structured decomposition of local image details from the source. Simultaneously, by combining shape complexity indicators and local feature entropy to generate cross-modal correction coefficients, the distribution complexity and dispersion of image visual features can be adaptively quantified, effectively solving the problem of misalignment and inaccurate matching between fine-grained visual features and condensed text commentary semantics, and improving the recognition accuracy of cross-modal entities such as three-point actions, breakthrough postures, and defensive behaviors. The semantic sampling constraint polygons are recursively and adaptively subdivided until the particle distribution uniformity of the sub-regions meets a preset threshold, avoiding feature redundancy or loss caused by fixed subdivision. By calculating the local feature entropy of each visual feature partition, the feature richness and discriminability of local regions can be objectively quantified, making the subsequent cross-modal alignment process more focused on high-information visual regions and improving the efficiency of multimodal data processing.

[0017] In a preferred embodiment of the present invention, step 1, acquiring athlete career profile data including image modal data and text modal data, converting the image modal data into visual semantic particle clusters, constructing a feature projection manifold based on the visual semantic particle clusters, and delineating semantic sampling constraint polygons on the feature projection manifold, may include: Step 101: Perform local feature detection on the image modal data to extract corner points, edge points, and color change points in the image to form an image feature point set. Specifically, this includes: traversing each pixel in the image, calculating the difference in grayscale value between the pixel and its 8 neighboring pixels, summing the absolute values ​​of the grayscale value differences to obtain the pixel gradient value of the pixel, with a preset gradient threshold between 10 and 50, filtering out pixels with pixel gradient values ​​greater than the preset gradient threshold, and then performing neighborhood verification on the filtered pixels. Retain pixels with pixel gradient values ​​greater than the preset gradient threshold and prominent positions within a 3×3 neighborhood centered on the current pixel as corner points. Focus on extracting corner points at key action positions in the image, such as wrist rotation during basketball shooting, fingertip force, body center of gravity shift, foot landing, and arm extension. Next, edge detection is performed on the athlete's motion image. Each pixel in the image is traversed, and the grayscale gradient values ​​in the horizontal and vertical directions are calculated. The grayscale gradient values ​​in the two directions are squared and added together. The square root of the sum is then taken to obtain the gradient amplitude of the pixel. A preset amplitude threshold is set between 15 and 60. Pixels with gradient amplitudes greater than the preset threshold are selected as edge points where pixel values ​​change abruptly. Emphasis is placed on extracting edge points formed by body contours, limb boundaries, and equipment edges in the image. Color feature analysis is then performed on the athlete's motion image, converting the image from the RGB color space to the HSV color space and calculating the gradient amplitude of each pixel. The hue, saturation, and brightness values ​​of each pixel are analyzed. For each pixel, the hue, saturation, and brightness differences are compared with those of its neighboring pixels. The squares of each difference are squared, and the sums are then taken as the square root to obtain the color difference value. A preset color threshold of 5 to 30 is used to filter out pixels with color difference values ​​greater than the preset threshold, identifying these as significant color abrupt changes. The focus is on extracting color abrupt changes between the jersey and the field, and between limbs and the background. All extracted corner points, edge points, and color abrupt changes are then compiled, and duplicate pixels with identical coordinates are removed to form a complete set of image feature points.

[0018] Step 102 involves performing spatial analysis on the image feature point set, merging adjacent feature points with similar feature attributes into the same visual semantic particle cluster, resulting in multiple visual semantic particle clusters. Specifically, this includes: conducting dual analysis of spatial location and feature attributes on all feature points in the image feature point set, sequentially traversing each feature point, calculating the spatial distance between the current feature point and each surrounding feature point (with a preset distance threshold of 3 to 15 pixels), identifying feature points with a spatial distance less than the preset distance threshold as adjacent feature points, comparing the pixel grayscale values ​​and color features of the current feature point with those of the surrounding feature points, identifying feature points with a feature value difference of less than 10 as feature points with similar feature attributes, merging multiple feature points that simultaneously satisfy both adjacent location and similar feature attributes into the same visual semantic particle cluster, so that shooting action-related features, breakthrough action-related features, and defensive action-related features are respectively clustered into independent sets, and traversing and classifying all feature points according to the above merging rules, ultimately obtaining multiple independent and non-overlapping visual semantic particle clusters.

[0019] Step 103: Calculate the centroid of each visual semantic particle cluster, forming a centroid point set in a high-dimensional space from all centroids; calculate the Euclidean distance between any two points in the centroid point set, constructing a distance matrix; based on the distance matrix, map the high-dimensional centroid point set to a low-dimensional space, minimizing the difference between the Euclidean distance between points in the low-dimensional space and the corresponding distance values ​​in the distance matrix, obtaining a coordinate point set in the low-dimensional space. Use this coordinate point set as a feature projection manifold, specifically including: for each visual semantic particle cluster, read the horizontal and vertical coordinates of all feature points within the cluster; sum the horizontal coordinates of all feature points to obtain the sum of horizontal coordinates; divide the sum of horizontal coordinates by the total number of feature points within the current particle cluster to obtain the average horizontal coordinate; sum the vertical coordinates of all feature points to obtain the sum of vertical coordinates; divide the sum of vertical coordinates by the total number of feature points within the current particle cluster to obtain the average vertical coordinate; combine the average horizontal coordinate and the average vertical coordinate as the centroid of the current visual semantic particle cluster; calculate the centroid of all visual semantic particle clusters in the above manner, forming a centroid from all centroids. The centroids form a set of centroid points in high-dimensional space. Traversing any two distinct centroids in this set, the horizontal and vertical coordinates of each centroid are read. The difference between the horizontal and vertical coordinates is obtained by subtracting the horizontal coordinates and the difference between the vertical coordinates. The horizontal and vertical coordinate differences are squared, and the sum of these squares is taken as the square root of the sum, yielding the straight-line distance between the two centroids in high-dimensional space. The straight-line distances between each pair of centroids are arranged in order of their index, constructing a complete distance matrix. Based on this distance matrix, the set of centroid points in high-dimensional space is mapped to a two-dimensional low-dimensional space. During this mapping process, the horizontal and vertical coordinates of each point in the low-dimensional space are continuously adjusted, and the straight-line distance between any two points in the low-dimensional space is continuously calculated. This straight-line distance is compared with the distance between the corresponding two points in the distance matrix, continuously reducing the numerical difference until it no longer decreases, resulting in a stable set of coordinate points in the low-dimensional space. This low-dimensional set of coordinate points is then identified as the feature projection manifold.

[0020] Step 104: In the low-dimensional space where the feature projection manifold is located, calculate the convex hull of all low-dimensional coordinate points, and determine the polygonal region enclosed by the convex hull boundary as the semantic sampling constraint polygon. Specifically, this includes: In the two-dimensional low-dimensional space where the feature projection manifold is located, perform a complete convex hull calculation for all low-dimensional coordinate points. First, traverse all low-dimensional coordinate points and find the coordinate point with the smallest horizontal coordinate value as the starting point of the convex hull. Then, starting from the starting point, select subsequent low-dimensional coordinate points in sequence. For any three consecutively selected coordinate points, calculate the vector formed by the first two coordinate points and the vector formed by the last two coordinate points. The turning relationship between the vectors is determined to determine whether the turn formed by the three coordinate points is a left turn or a right turn. The coordinate points that form a left turn are retained, and the coordinate points that form a right turn are removed. The above turning judgment and coordinate point filtering process is repeated until the starting point of the convex hull is returned. All the outermost low-dimensional coordinate points that have been filtered and retained are connected in sequence to form the smallest convex polygon boundary that can completely enclose all low-dimensional coordinate points. This boundary can completely enclose the projection points corresponding to all visual features in the athlete's motion image. The closed continuous polygon region enclosed by the convex hull boundary is determined as the semantic sampling constraint polygon.

[0021] This embodiment extracts multiple types of local features, including corner points, edge points, and color change points, from image modal data. This allows for the complete preservation of key details such as wrist posture, body center of gravity, foot position, and arm extension in athlete motion images, preventing the loss of crucial motion information. Furthermore, by performing spatial and attribute-based dual clustering of feature points to form visual semantic particle clusters, it enables the semantic aggregation of scattered visual features, enhancing the overall integrity of the features.

[0022] In a preferred embodiment of the present invention, step 2 involves calculating the shape complexity index of the semantic sampling constraint polygon. The shape complexity index is quantified by analyzing the curvature changes of the polygon's boundary points and the ratio of its area to its perimeter. Based on the spatial distribution characteristics of the visual semantic particle clusters, the semantic sampling constraint polygon is subdivided using the shape complexity index to generate multiple visual feature partitions. The visual semantic particle clusters are mapped to the corresponding visual feature partitions, and the local feature entropy of the visual feature partitions is calculated. The local feature entropy and the shape complexity index are fused to generate cross-modal correction coefficients, which may include: Step 201: Extract the coordinates of all boundary points from the boundary of the semantic sampling constraint polygon, and sort the coordinate points in a clockwise direction to obtain a boundary point sequence. Specifically, this includes: extracting the horizontal and vertical coordinates of all boundary points from the boundary of the semantic sampling constraint polygon; starting from any boundary point, determining the next boundary point in a clockwise direction; sorting the coordinates of all boundary points in sequence; and combining the sorted boundary points into an ordered boundary point sequence.

[0023] Step 202: Traverse each boundary point in the boundary point sequence. Based on the direction of the line segment formed by the current point, the previous point, and the next point, calculate the local curvature change at the current point. Accumulate the local curvature changes of all boundary points to obtain the overall curvature change. Calculate the area and perimeter of the semantic sampling constraint polygon to obtain the ratio of area to perimeter. Specifically, this includes: traversing each boundary point in the boundary point sequence; for the currently processed boundary point, reading the previous and next boundary points in the boundary point sequence, obtaining the horizontal and vertical coordinates of the previous, current, and next boundary points respectively; determining the extension direction of the first line segment based on the coordinate difference between the previous and current boundary points; determining the extension direction of the second line segment based on the coordinate difference between the current and next boundary points; placing the first and second line segments at the same coordinate origin; calculating the angle between the two line segments; dividing the angle by 180° to obtain the local curvature change at the current boundary point position; and calculating the local curvature change of all boundaries in the boundary point sequence in the above manner. The local curvature change of a point is calculated by summing the local curvature changes of all boundary points sequentially, and the sum is taken as the overall curvature change. Simultaneously, based on the boundary point coordinates of the semantically sampled constrained polygon, the area enclosed by the polygon is calculated using the shoelace formula. Specifically, the calculation process involves sequentially reading the horizontal and vertical coordinates of each boundary point in the boundary point sequence, multiplying the horizontal coordinate of the current boundary point by the vertical coordinate of the next boundary point to obtain the first product, multiplying the vertical coordinate of the current boundary point by the horizontal coordinate of the next boundary point to obtain the second product, subtracting the second product from the first product to obtain the difference term corresponding to the current boundary point, and repeating this process for all boundary points. All difference terms are then summed to obtain the total difference. The absolute value of the total difference is divided by 2 to obtain the area enclosed by the semantically sampled constrained polygon. Next, the straight-line distances between adjacent boundary points are calculated sequentially, and the sum of these straight-line distances is used to obtain the perimeter of the semantically sampled constrained polygon boundary. The calculated area value is divided by the perimeter value to obtain the ratio of area to perimeter.

[0024] Step 203: Normalize the overall curvature change and the ratio of area to perimeter. Merge the normalized overall curvature change and the normalized ratio of area to perimeter, using the fusion result as a shape complexity index. Specifically, this includes: setting the maximum curvature change range to 1.5 times the total number of boundary points; dividing the overall curvature change by this maximum curvature change range to complete the normalization process, ensuring the overall curvature change falls within the range of 0 to 1; finding the smallest bounding rectangle that completely encloses the semantic sampling constraint polygon; and calculating the area and perimeter of this bounding rectangle. The area-to-perimeter ratio of the circumscribed rectangle is obtained by dividing the area of ​​the circumscribed rectangle by its perimeter. The maximum area-to-perimeter ratio is preset to be 1.2 times the maximum area-to-perimeter ratio of the circumscribed rectangle. The area-to-perimeter ratio of the semantic sampling constraint polygon is divided by this maximum area-to-perimeter ratio to complete the normalization process, so that the area-to-perimeter ratio falls within the range of 0 to 1. The normalized overall curvature change is multiplied by 0.6, and the normalized area-to-perimeter ratio is multiplied by 0.4. The two multiplication results are added together to complete the weighted fusion. The fused result is used as the shape complexity index of the semantic sampling constraint polygon.

[0025] Step 204: Based on the spatial distribution density of visual semantic particle clusters within the semantic sampling constraint polygon, determine the priority region for subdivision and the subdivision granularity based on the shape complexity index; recursively subdivide the semantic sampling constraint polygon according to the priority region and subdivision granularity, with each subdivision dividing the current region into multiple sub-regions; during the recursive subdivision process, check the number of visual semantic particle clusters contained in each sub-region, calculate the range between the number of visual semantic particle clusters in all sub-regions, compare the range with a preset uniformity threshold, and stop the recursive subdivision when the range is less than the preset uniformity threshold. All sub-regions obtained at the point of stopping the subdivision are used as visual feature partitions. Specifically, this includes: counting the distribution position of each visual semantic particle cluster within the semantic sampling constraint polygon, dividing the semantic sampling constraint polygon into several unit regions of equal size, counting the number of visual semantic particle clusters in each unit region, obtaining the spatial distribution density of the corresponding unit region, and subdividing regions with higher distribution densities. The region is determined as the priority segmentation region. The segmentation granularity is determined based on the value of the shape complexity index. The larger the shape complexity index value, the finer the segmentation granularity and the more sub-regions are divided. According to the determined priority segmentation region and segmentation granularity, the semantic sampling constraint polygon is recursively segmented. Each segmentation divides the currently processed region into multiple uniformly sized sub-regions along the center position. The preset uniformity threshold is an integer between 2 and 4. During the entire process of recursive segmentation, the number of visual semantic particle clusters contained in each sub-region is counted one by one. The number of particle clusters in all sub-regions is traversed, and the maximum and minimum values ​​are found. The maximum value is subtracted from the minimum value to obtain the range value. The range value is compared with the preset uniformity threshold. If the range value is greater than or equal to the preset uniformity threshold, the current sub-region continues to be recursively segmented. If the range value is less than the preset uniformity threshold, the recursive segmentation of the current region is stopped. All sub-regions obtained after stopping the segmentation of all regions are used as visual feature partitions.

[0026] Step 205: Based on the spatial coordinates of each visual semantic particle cluster, assign each visual semantic particle cluster to the visual feature partition it falls into, obtaining the set of visual semantic particle clusters corresponding to each visual feature partition; for each visual feature partition, based on the set of visual semantic particle clusters corresponding to the visual feature partition, count the total number of visual semantic particle clusters within the visual feature partition, calculate the distribution probability of each visual semantic particle cluster within the visual feature partition, calculate the information entropy of the visual feature partition based on the distribution probability, and use the calculated information entropy as the local feature entropy of the visual feature partition. Specifically, this includes: reading the horizontal and vertical coordinates of each visual semantic particle cluster in low-dimensional space, determining the visual feature partition into which the visual semantic particle cluster falls based on the coordinate position, and assigning each visual semantic particle cluster to the corresponding visual feature partition, so that the shooting action particle cluster, the breakthrough action particle cluster, and the other visual semantic particle clusters fall into the corresponding visual feature partition. The particle clusters for action and the particle clusters for defensive actions are each placed into their respective partitions, resulting in a set of visual semantic particle clusters for each visual feature partition. For each visual feature partition, the total number of visual semantic particle clusters within that partition is counted. The number of feature points of all visual semantic particle clusters within the current visual feature partition is read and summed to obtain the total number of feature points. For each visual semantic particle cluster in the visual feature partition, the number of feature points within that cluster is divided by the total number of feature points in the current visual feature partition to obtain the probability distribution of the current visual semantic particle cluster. The probability distribution of all visual semantic particle clusters within the visual feature partition is then calculated using the natural logarithm. Each probability distribution is multiplied by its corresponding natural logarithm, and all multiplication results are summed to obtain the information entropy of the current partition. The calculated information entropy is used as the local feature entropy of the current visual feature partition.

[0027] Step 206: Summarize the local feature entropy corresponding to all visual feature partitions, calculate the arithmetic mean of all local feature entropies, and weight the arithmetic mean with the shape complexity index to obtain the cross-modal correction coefficient. Specifically, this includes: collecting the local feature entropy corresponding to all visual feature partitions, summing all local feature entropies sequentially to obtain the sum of local feature entropies, and dividing the sum of local feature entropies by the total number of visual feature partitions to obtain the arithmetic mean of all local feature entropies. Based on the core requirement of cross-modal semantic deviation correction (prioritizing the semantic integrity of matching feature distributions, supplemented by geometric shape correction), the weight range of the local feature entropy arithmetic mean is preset to be 0.6 to 0.8, and the weight range of the shape complexity index is preset to be 0.2 to 0.4. In this method, the weight of the local feature entropy arithmetic mean is selected as 0.7, and the weight of the shape complexity index is selected as 0.3. The arithmetic mean is multiplied by 0.7, the shape complexity index is multiplied by 0.3, and the two multiplication results are added to complete the weighting process. The final result after weighting is determined as the cross-modal correction coefficient.

[0028] In this embodiment, by calculating the distribution probability of particle clusters within each visual feature partition, the local feature entropy is obtained, which can accurately reflect the dispersion and richness of visual features within each partition. The arithmetic mean of the local feature entropy and the shape complexity index are weighted to generate cross-modal correction coefficients, which can combine geometric complexity and feature distribution characteristics to effectively correct the semantic correspondence deviation between image modality and text modality, and improve the accuracy and consistency of data alignment between the two modalities.

[0029] In a preferred embodiment of the present invention, step 3, based on cross-modal correction coefficients, inputs image modal data and text modal data into a pre-trained single- and dual-stream interaction model for semantic similarity alignment, and optimizes the alignment process using cross-modal correction coefficients to obtain a cross-modal entity set, may include: Step 301: The attention weights in the pre-trained single-stream / dual-stream interaction model are redistributed based on the cross-modal correction coefficients. The model obtained after redistributing the attention weights is used as the optimized single-stream / dual-stream interaction model. Specifically, this includes: defining the initial attention weight distribution of the pre-trained single-stream / dual-stream interaction model. This model includes an image encoding branch and a text encoding branch, both of which have attention mechanisms. The initial attention weights are equally distributed, with initial weight values ​​ranging from 0.05 to 0.1. That is, the attention weights of each feature channel in the image encoding branch are the same, all between 0.05 and 0.1, and the attention weights of each word feature in the text encoding branch are also the same, also between 0.05 and 0.1. The cross-modal correction coefficients obtained in step 206 are read. These coefficients range from 0.8 to 1.5 and are used to correct the semantic deviation between image action details and text commentary descriptions. The larger the coefficient value, the greater the semantic deviation between the image and the text, requiring more optimization of the attention weights. To improve alignment accuracy, the specific process of attention weight redistribution is as follows: The cross-modal correction coefficient is multiplied by the initial attention weight of the image coding branch to obtain the adjusted attention weight of the image coding branch, with an adjusted weight range of 0.04 to 0.15. The cross-modal correction coefficient is then multiplied by the initial attention weight of the text coding branch to obtain the adjusted attention weight of the text coding branch, also with an adjusted weight range of 0.04 to 0.15. Simultaneously, the adjusted attention weights of the two branches are normalized to ensure that the sum of the attention weights of each branch is 1. The normalization process involves dividing the adjusted attention weight of each branch by the sum of all attention weights in that branch to obtain the final attention weight, with a final attention weight range of 0.03 to 0.2, and the sum of all final attention weights within a single branch is 1. The model obtained after redistributing the attention weights is used as the optimized single-to-dual-stream interaction model.

[0030] Step 302: Input the image modal data into the image coding branch of the optimized single-dual-stream interaction model for global feature extraction to obtain image feature vectors; input the text modal data into the text coding branch of the optimized single-dual-stream interaction model for semantic feature extraction to obtain text feature vectors. Specifically, the image coding branch adopts a convolutional neural network structure, first preprocessing the input image modal data, adjusting the size of the image modal data to a uniform size to ensure consistent image resolution, specifically adjusting it to 256 pixels by 256 pixels, and then performing grayscale processing on the adjusted image. Normalization processing converts the grayscale values ​​of the image into values ​​between 0 and 1, reducing the impact of illumination changes on feature extraction. Specifically, normalization divides the grayscale value of each pixel by 255 to obtain the normalized grayscale value. After preprocessing, the image is input into the convolutional layer of a convolutional neural network. Feature extraction is performed using 3x3 convolutional kernels. After each convolution operation, the ReLU activation function is applied to enhance the non-linear expressive power of the features. Through multiple rounds of convolution and pooling operations, local and global features of the image are gradually extracted. The pooling operation... The algorithm employs max pooling, and the specific implementation process is as follows: Pooling operations correspond one-to-one with convolutional layers. After each round of convolution and ReLU activation, a max pooling operation is immediately performed. The pooling kernel size is fixed at 2x2, and the pooling stride is set to 2. Effective padding is used, meaning no boundary padding is performed, ensuring that the feature map size shrinks uniformly after pooling. Specifically, after each max pooling operation, the width and height of the feature map are reduced to half their original values, while the number of channels remains consistent with the output channel number of the previous convolutional layer. The core operation of max pooling is the 2x2... Within the pooling window, the four pixels within the window are traversed, and the pixel with the largest value is selected as the output value of the pooling window, while the other three pixel values ​​are discarded. This method preserves the most critical and representative feature information in the image, effectively filtering redundant features, reducing feature dimensionality, decreasing model computation, and preventing overfitting. After each round of max pooling, the output feature map is fed into the next convolutional layer for further feature extraction. After three rounds of alternating convolution and max pooling operations, a high-dimensional feature map containing both local and global image features is obtained. Finally, a fully connected layer converts the extracted global features into a fixed-dimensional vector, which is the image feature vector, with the vector dimension set to 512 to 1024 dimensions.

[0031] The text encoding branch adopts a recurrent neural network structure. First, the input text modal data is preprocessed to remove invalid characters and redundant information. Invalid characters include special symbols, punctuation marks, and meaningless auxiliary words. Redundant information includes repetitive descriptions and modifiers. The text is then split into individual words according to a preset word segmentation rule. The word segmentation rule is tailored to the characteristics of sports technical commentary text, prioritizing the segmentation of action keywords and body part keywords. Each word is then converted into a corresponding word vector. The word vector dimension is consistent with the image feature vector dimension, i.e., 512 to 1024 dimensions. After preprocessing, the word vector sequence is input into the recurrent neural network. The recurrent layer captures the contextual semantic relationships in the text. The number of neurons in the hidden layer of the recurrent layer is set to 256 to 512. The semantic features of the text are extracted step by step. Finally, the extracted semantic features are converted into fixed-dimensional text feature vectors through a fully connected layer.

[0032] Step 303: Input the image feature vector and text feature vector into the interaction fusion layer of the optimized single-dual-stream interaction model, calculate the cosine similarity between the image feature vector and the text feature vector to obtain a semantic similarity score set; extract the image feature vector and text feature vector combination with similarity scores exceeding the preset matching threshold from the semantic similarity score set to generate candidate alignment pairs, specifically including: for each image feature vector, calculate the cosine similarity with each text feature vector. The calculation first calculates the dot product of the image feature vector and the text feature vector. The dot product is calculated by multiplying each element in the image feature vector with the corresponding element in the text feature vector, and then summing all the multiplication results to obtain the dot product result; then calculate the magnitude of the image feature vector and the magnitude of the text feature vector respectively. The magnitude is calculated by squaring each element in the vector, and then summing all the squared results. The square root operation is performed to obtain the magnitude of the vector. The dot product result is divided by the product of the magnitudes of the image feature vector and the text feature vector to obtain the cosine similarity score of the image feature vector and the text feature vector. The similarity score ranges from 0 to 1. The closer the score is to 1, the higher the semantic matching degree between the image feature and the text feature. The closer the score is to 0, the lower the semantic matching degree. The cosine similarity scores of all combinations of image feature vectors and text feature vectors are calculated in the above manner. All scores are summarized to obtain the semantic similarity score set. A confidence matching threshold is preset. This threshold is set according to the matching characteristics of sports athlete action images and technical commentary texts, and the value ranges from 0.6 to 0.8. Image feature vectors and text feature vector combinations with similarity scores exceeding the preset matching threshold are extracted from the semantic similarity score set. Each combination corresponds to a set of image modality data and text modality data. These combinations are used to generate candidate alignment pairs.

[0033] Step 304: Extract local regions from the image modal data corresponding to the candidate alignment pairs, extract key phrases from the text modal data corresponding to the candidate alignment pairs, and combine the local regions and key phrases to obtain preliminary cross-modal entities; aggregate and deduplicate all preliminary cross-modal entities to obtain a cross-modal entity set, specifically including: locating key regions related to text features in the image modal data based on the feature positions corresponding to the image feature vectors in the candidate alignment pairs. These key regions are the core parts of the athlete's movements; for example, if the text features correspond to a shooting action, then key parts such as the wrist and fingertips in the image are located. A rectangular bounding box is used to extract the local region of this key part. The specific calculation process is as follows: calculation is performed based on the coordinates of the center point of the located key part. Let the coordinates of the center point of the key part be (…). , The coordinates are obtained by mapping the feature positions corresponding to the image feature vectors. The mapping calculation method is to map the feature indices of the image feature vectors to the image pixel coordinates one-to-one using a linear mapping formula. First, obtain the pixel coordinates of the center point of the key part; second, determine the initial size of the rectangle. Based on the actual size of the key part, the width and height of the initial rectangle are calculated according to the outer rectangle size of the key part. The outer rectangle size is obtained by iterating through all pixels of the key part, calculating the difference between the maximum and minimum values ​​of the pixel's horizontal coordinate to get the width, and the difference between the maximum and minimum values ​​of the pixel's vertical coordinate to get the height, ensuring that the initial rectangle can completely encompass the key part; according to the preset rectangle size range (64 pixels × 64 pixels to 256 pixels × 256 pixels), the initial rectangle size is adjusted. If the initial rectangle width or height is less than 64 pixels, the corresponding dimension is adjusted to 64 pixels; if it is greater than 256 pixels, it is adjusted to 256 pixels; if it is within the range, the initial size is maintained; with the center point ( , Using the adjusted rectangle as the center, calculate the coordinates of its four vertices. , , , The bounding box is determined based on the coordinates of the four vertices to extract local regions, ensuring that each region fully encompasses the key motion features. All local regions are then uniformly sized to 128 pixels by 128 pixels for easier subsequent entity processing. Key phrases are extracted from the text modal data corresponding to the candidate alignment pairs. Specifically, semantic analysis is performed on the text modal data to extract word combinations that represent the core information of the athlete's movements, removing redundant descriptive information. For example, if the text modal data states that the wrist rotation angle is reasonable during a basketball shot, the extracted key phrase is "fist rotation during a basketball shot," ensuring that the key phrases accurately correspond to the motion features of the local image regions. The extracted local regions are combined with key phrases, and each combination constitutes a preliminary cross-modal entity. The preliminary cross-modal entity includes both the action key region of the image modality and the action key description of the text modality. All preliminary cross-modal entities are aggregated and deduplicated. The aggregation process involves grouping preliminary cross-modal entities with the same action features and text descriptions into one category. The deduplication process involves reading each preliminary cross-modal entity one by one and determining whether the combination of the local region and key phrase of the entity already exists. If it exists, the entity is removed; if it does not exist, the entity is retained. After traversing all preliminary cross-modal entities to complete the aggregation and deduplication process, a set of cross-modal entities is obtained.

[0034] Detailed construction, training, and implementation process of the single-stream / dual-stream interaction model: The construction process of the single- and dual-stream interaction model involves building a complete model structure that includes image encoding branches, text encoding branches, and an interaction fusion layer. The overall model adopts an end-to-end architecture to ensure that each branch works collaboratively and that data transmission is smooth. The image encoding branch employs a convolutional neural network architecture, consisting of 5 convolutional layers, 3 pooling layers, and 2 fully connected layers. The number of convolutional kernels in the convolutional layers are 32, 64, 128, 256, and 512 respectively, all using 3x3 kernels with a stride of 1 and same padding to ensure the image size remains unchanged after convolution. The 3 pooling layers all use 2x2 max-pooling kernels with a stride of 2 to reduce feature dimensionality and retain key features. The 2 fully connected layers have 1024 and 512 neurons respectively. These fully connected layers convert the extracted features into fixed-dimensional image feature vectors, using ReLU activation functions. The last fully connected layer does not have an activation function and directly outputs the image feature vector. The text encoding branch employs a recurrent neural network architecture, consisting of 3 recurrent layers and 2 fully connected layers. The recurrent layers use ReLU activation functions. The network uses a Long Short-Term Memory (LSTM) structure. Each recurrent layer has 256 to 512 hidden neurons. Dropout regularization is applied to the recurrent layers, with a dropout probability ranging from 0.2 to 0.3, to prevent overfitting and capture the contextual semantic relationships of the text. The two fully connected layers have 512 and 512 to 1024 neurons respectively, both using ReLU activation. The last fully connected layer outputs a fixed-dimensional text feature vector to ensure consistency with the image feature vector dimension. The interactive fusion layer uses a fully connected structure, consisting of one fully connected layer and one output layer. The fully connected layer has 512 neurons and uses ReLU activation to fuse the image and text feature vectors. The output layer calculates cosine similarity and outputs a semantic similarity score ranging from 0 to 1.

[0035] The training process of the single-stream / dual-stream interactive model is as follows: Image data of athletes' competition movements and corresponding technical commentary text data are selected as the training dataset. This dataset covers various athlete movements such as shooting, dribbling, and defense. The image data resolution is uniformly adjusted to 256 pixels by 256 pixels, and the text data undergoes word segmentation and stop word removal. The total dataset contains no fewer than 10,000 sets. The training dataset is divided into a training set and a validation set in an 8:2 ratio. The training set is used for model parameter training, and the validation set is used to verify the model training effect. Before training, the model parameters are initialized. Convolutional and fully connected layer parameters are initialized using the Xavier initialization method, and recurrent layer parameters are initialized using the orthogonal initialization method. The model learning rate is set to 0.001 to 0.0001. An adaptive moment estimation optimization algorithm is used for parameter optimization, with the momentum parameter set to 0.9 and the weight decay coefficient set to 0. To prevent overfitting, during training, image modal data and text modal data are input into the image encoding branch and text encoding branch respectively. The corresponding feature vectors are extracted, and the semantic similarity score is calculated through the interactive fusion layer. The cross-entropy loss function is used to calculate the error between the similarity score output by the model and the actual matching label. The actual matching label is set to 0 or 1 according to the actual matching degree between the image and the text. If they match, the label is 1; if they do not match, the label is 0. The parameters of the convolutional kernel, recurrent layer, and fully connected layer of the model are adjusted through the backpropagation algorithm to minimize the error value. The training is iterated for 150 to 250 rounds. After each round of training, the alignment accuracy of the model is verified using a validation set. The learning rate is adjusted every 10 rounds with a learning rate adjustment coefficient of 0.9. Training is stopped when the alignment accuracy of the validation set reaches more than 90% and there is no significant improvement for 5 consecutive rounds, thus obtaining the pre-trained single- and dual-stream interactive model.

[0036] The implementation process of the single- and dual-stream interaction model is as follows: The pre-trained single- and dual-stream interaction model is deployed to a server, and the model runtime environment is configured to ensure that the server's GPU memory is no less than 16GB to support efficient model operation. The model input consists of image modal data and text modal data. The image modal data needs to be preprocessed to a uniform size and grayscale range, and the text modal data needs to be preprocessed into word vector sequences after word segmentation. The batch size of the input data is set to 32 to 64 groups to ensure model efficiency and processing accuracy. During model runtime, the image encoding branch extracts features from the input image modal data and outputs image feature vectors, while the text encoding branch extracts features from the input text modal data and outputs text feature vectors. The interaction fusion layer fuses the two feature vectors and calculates cosine similarity, outputting a semantic similarity score to provide data support for the subsequent generation of candidate alignment pairs. Subsequently, the model's attention weights are optimized through cross-modal correction coefficients to further improve alignment accuracy.

[0037] This embodiment optimizes the attention weights of the single- and dual-stream interaction model by using cross-modal correction coefficients, which can specifically correct semantic deviations between images and text, improve the semantic similarity alignment accuracy between image modal data and text modal data, and ensure the matching accuracy of candidate alignment pairs.

[0038] In a preferred embodiment of the present invention, step 4, constructing a set of relationships between entities based on the cross-modal entity set, combined with predefined domain rules and a deep pre-trained language model, may include: Step 401: Perform type recognition on each entity in the cross-modal entity set, determine the entity category based on the image modal data and text modal data corresponding to the entity, and obtain an entity set with category labels; based on the entity set with category labels, extract rule matching conditions that match the entity category from the predefined domain rule base, input the entity set with category labels into the rule matching conditions for matching, and obtain the first relation candidate set. Specifically, this includes: reading each entity in the cross-modal entity set one by one. Each entity contains corresponding image modal data and text modal data, where the image modal data is a local area image related to the athlete's competition action, and the text modal data is a key phrase for technical commentary on the corresponding action. For each entity, we first analyze its corresponding text modal key phrases and extract the core keywords in the phrases. For example, keywords such as shooting, wrist rotation, and fingertip power in the key phrases for shooting actions, defensive positioning and arm interception in the key phrases for defensive actions, and breakthrough direction and footwork in the key phrases for breakthrough actions. Then, we analyze its corresponding image modal local regions and identify the athlete's action features contained in the local regions. For example, if the local region is the wrist, it corresponds to shooting or passing actions, and if the local region is the leg, it corresponds to footwork or breakthrough actions. Combining text keywords and image action features, the category of each entity is determined. Entity categories are specifically divided into five types: shooting action entities, dribbling action entities, defensive action entities, body part entities, and equipment entities. Each entity is associated with its corresponding category label, resulting in a set of entities with category labels. Based on the set of entities with category labels, matching rules are extracted from a predefined domain rule base. This predefined domain rule base is constructed based on the standards for commenting on athletes' movement techniques and contains association rules between various entity categories. For example, there are association rules between shooting action entities and wrist parts entities, shooting action entities and fingertip parts entities, and so on. There are association rules between entities such as the shooting action entity and the wrist part entity. Each rule corresponds to a specific rule matching condition. The rule matching condition is the feature matching requirement of two different categories of entities. For example, the matching condition for the shooting action entity and the wrist part entity is that the text keyword of the shooting action entity contains wrist rotation and the local area of ​​the image contains the wrist part. Each entity in the entity set with category labels is compared with the category requirements in the rule matching condition. Two entities that meet the rule matching condition are selected. These two entities are combined with the corresponding association rule to obtain the first relationship candidate set. Each item in the first relationship candidate set contains two entities and the association relationship type between the two entities.

[0039] Step 402: Entities containing text modal data are selected from the entity set with category labels. Based on the text modal data corresponding to the selected entities, text fragments containing at least two selected entities are located in the original text corpus. These located text fragments are used as context windows. The context windows are then input into a deep pre-trained language model for analysis to identify semantic relationships between entities in the text fragments, resulting in a second set of candidate relations. Specifically, this involves: first, selecting entities containing text modal data from the entity set with category labels. All cross-modal entities contain text modal data; therefore, the selected entities are all entities in the entity set with category labels. For each selected entity, its corresponding text modal data is read. Based on the keywords of this text modal data, the original text corpus is searched and located. The original text corpus is the complete content of technical commentary text in a sports athlete's career file. Text fragments containing at least two text keywords corresponding to the selected entities are located in the original text corpus. These text fragments are used as context windows. The length of the context window is determined by its ability to fully contain the relational description of the two entities, typically 50 to 200 characters. All the localized context windows are input one by one into a deep pre-trained language model for analysis. The deep pre-trained language model is first constructed, trained, and implemented in detail. The construction process involves selecting a bidirectional encoder representation model as the basic model architecture, building a complete model structure including an input layer, encoder, and output layer. The input layer uses an embedding layer structure, including word embeddings and position embeddings. The word embedding dimension is set to 768 dimensions, and the position embedding dimension is also 768 dimensions. This layer receives the context window text data, splits the text data into individual words according to preset word segmentation rules, converts each word into a fixed-dimensional word vector, and simultaneously adds position embedding information to capture the positional association features of the text. The encoder contains a 12-layer transformer encoder, each layer of which includes a multi-head attention mechanism and a fully connected layer. The multi-head attention mechanism is designed... The system is configured with 12 heads, each with a feature dimension of 64, for a total feature dimension of 768. This is used to capture the semantic relationships between different entities in the text, enabling simultaneous attention to word associations at different locations within the text. The fully connected layer contains two linear transformations. The first linear transformation converts the feature dimension output by the attention mechanism to 3072 dimensions, followed by activation processing using the ReLU activation function. The second linear transformation restores the feature dimension to 768 dimensions, optimizing the output features of the attention mechanism. Layer normalization is added between each transformer encoder, with the normalization minimum set to 0.00001 to prevent numerical overflow. The output layer uses the softmax activation function, with the number of neurons matching the number of entity relationship types. It outputs the probability of the relationship type between entities, with a probability value ranging from 0 to 1. A higher probability value indicates a higher matching degree for that relationship type.

[0040] The training process of the deep pre-trained language model is as follows: Select technical commentary texts of sports athletes as the training dataset. These texts are all from the technical commentary records in the career archives of sports athletes, covering commentary on various actions such as shooting, breakthrough, and defense. The total number of texts is no less than 50,000, and the length of each text is controlled between 50 and 200 characters. The texts are preprocessed by word segmentation, stop word removal, entity annotation, and entity relationship annotation to obtain the annotated training dataset. The training dataset is divided into a training set and a validation set in an 8:2 ratio. The training set is used to train the model parameters, and the validation set is used to verify the training effect of the model. Before training, the model parameters were initialized. Xavier initialization was used for the embedding and fully connected layers, while orthogonal initialization was used for the attention mechanism parameters. The learning rate was set to 0.0001 to 0.00001. An adaptive moment estimation (IME) algorithm was used for parameter optimization, with a momentum parameter set to 0.9 and a weight decay coefficient set to 0.0001. Dropout regularization was applied, with dropout probabilities ranging from 0.1 to 0.2 to prevent overfitting. During training, the context window text in the training set was used as input, and the actual relationship types between entities in the text were used as labels. These labels were represented using one-hot encoding. The model parameters were adjusted using backpropagation, and the error between the model's output relationship type probability and the actual labels was calculated using the cross-entropy loss function. This error was minimized. The model is trained iteratively for 100 to 200 rounds. After each round, the model accuracy is verified using a validation set. The matching ratio between the model's predicted relation types and the actual relation types in the validation set is calculated. The learning rate is adjusted every 10 rounds with a learning rate adjustment coefficient of 0.9. Training stops when the validation set accuracy reaches 90% or higher and there is no significant improvement for 5 consecutive rounds, resulting in a trained deep pre-trained language model. The implementation process of the deep pre-trained language model is as follows: the trained model is deployed to a server, the model's operating environment is configured, and the server memory is ensured to be no less than 32GB to support efficient model inference. The model input is preprocessed context window text, which must be processed according to the word segmentation rules required by the model. The batch processing size is set to 16 to 32 groups to ensure model inference efficiency and recognition accuracy. During model execution, the input layer converts text into word vectors and position embedding vectors. The encoder captures semantic relationships between entities in the text through a multi-head attention mechanism, optimizes feature representation through a fully connected layer, and outputs the probability of various relationship types between entities. The relationship type with the highest probability is selected as the semantic relationship between entities. For example, in a text segment, the two entities corresponding to the wrist rotation force in a shooting action are the shooting action entity and the wrist part entity. The semantic relationship is the action-dependent part. The two entities are combined with the identified semantic relationship type to obtain the second relationship candidate set. Each item in the second relationship candidate set contains two entities and the semantic relationship type between the two entities.

[0041] Step 403: Merge the first candidate relation set and the second candidate relation set to remove duplicates, resulting in a deduplicated candidate relation set. Specifically, this involves merging the first candidate relation set and the second candidate relation set by summarizing all items in both sets into a temporary set, and then performing deduplication on the temporary set. The deduplication process is as follows: Read each item in the temporary set one by one. Each item contains two entities and one relation type. Determine whether the combination of the two entities and relation type in the current item already exists in the temporary set. If it exists, remove the current item; if it does not exist, keep the current item. After traversing all items in the temporary set to complete the deduplication, the deduplicated candidate relation set is obtained.

[0042] Step 404: Evaluate the confidence level of the deduplicated candidate relation set, and select entity pairs and relation type combinations whose confidence levels exceed a preset confidence threshold. These selected combinations are then used as the relation set between entities. Specifically, this includes: calculating the confidence score for each item in the deduplicated candidate relation set; the confidence score is calculated using two dimensions. The first dimension is rule matching confidence: if the item comes from the first candidate relation set, the rule matching confidence is 1; if the item comes from the second candidate relation set, the rule matching confidence is 0.8. The rule matching confidence value is fixed at 0.8 or 1. The second dimension is semantic matching confidence, which is output by a deep pre-trained language model. The model evaluates each... The semantic matching probability corresponding to a candidate relation is output, which is the semantic matching confidence score, ranging from 0 to 1. The confidence scores of rule matching and semantic matching are weighted and summed, with a weighting ratio of 0.4 for rule matching confidence and 0.6 for semantic matching confidence. Specifically, the confidence score is equal to the confidence score of rule matching multiplied by 0.4 plus the confidence score of semantic matching multiplied by 0.6, with a confidence score ranging from 0.32 to 1. A pre-set confidence threshold is set, ranging from 0.7 to 0.8. The confidence score of each candidate relation is compared with the pre-set confidence threshold, and entity pairs with confidence scores exceeding 0.7 are selected as the relation type combination, and the selected combination is taken as the relation set between entities.

[0043] In this embodiment, the construction and training of a deep pre-trained language model, combined with corpus of technical commentary from sports athletes, can accurately identify semantic relationships between entities in text, compensate for the limitations of domain rules, enrich the types of entity relationships, and improve the completeness of the relationship set.

[0044] In a preferred embodiment of the present invention, step 5, storing entities as nodes and relationships as edges in a graph database based on the cross-modal entity set and the set of relationships between entities to construct a knowledge graph of an athlete's career profile, may include: Step 501 involves assigning a unique node identifier to each entity in the cross-modal entity set, determining the node's type attribute based on the entity's corresponding category label, extracting the node's descriptive information from the entity's corresponding image modal data and text modal data, and combining the node identifier, type attribute, and descriptive information into the entity node's storage format to obtain the entity node set to be stored. Specifically, this includes: assigning a unique node identifier to each entity in the cross-modal entity set, with the node identifier using a combination of numbers and letters, each identifier being 10 characters long to ensure that node identifiers for different entities are not duplicated; for each entity, determining the node's type attribute based on its corresponding category label, for example, the type attribute of a shooting action entity is "shooting action," the type attribute of a wrist part entity is "body part," and the type attribute of a basketball equipment entity is "equipment"; then, extracting the node's descriptive information from the entity's corresponding image modal data and text modal data, where the image modal data's descriptive information is the specific content of a local area of ​​the image, such as a partial image of a wrist rotation action, and the text modal data's descriptive information is the complete content of key phrases, such as "the wrist rotation angle is reasonable when shooting." The node identifier, type attribute, and description information are combined into the storage format of the entity node. The storage format is arranged in the order of node identifier first, type attribute in the middle, and description information last to ensure that the storage format of each entity node is consistent. The storage formats of all entity nodes are summarized to obtain the set of entity nodes to be stored.

[0045] Step 502: Assign a unique relation identifier to each entity pair and relation type combination in the entity relationship set. Based on the two entities in the entity pair, search for the corresponding node identifiers in the entity node set to be stored. Use the two found node identifiers as the start node identifier and end node identifier, respectively. Determine the relation type label based on the relation type corresponding to the entity pair. Combine the relation identifier, start node identifier, end node identifier, and relation type label into the storage format of the relation edge to obtain the set of relation edges to be stored. Specifically, this includes: assigning a unique relation identifier to each entity pair and relation type combination in the entity relationship set. The relation identifier uses the same format as the node identifier, with a length of 10 characters, ensuring that relation identifiers for different relations are not duplicated; for each entity pair and relation type combination, read the entity... For each of the two entities, the corresponding node identifiers are searched in the set of entity nodes to be stored. The node identifier corresponding to the first entity is used as the starting node identifier, and the node identifier corresponding to the second entity is used as the ending node identifier. The relation type label is determined according to the relation type corresponding to the entity pair. For example, if the entity pair is a shooting action entity and a wrist part entity, and the relation type is action-dependent part, then the relation type label is action-dependent part. The relation identifier, starting node identifier, ending node identifier, and relation type label are combined into the storage format of the relation edge. The storage format is arranged in the order of relation identifier first, starting node identifier second, ending node identifier third, and relation type label last to ensure that the storage format of each relation edge is consistent. The storage formats of all relation edges are summarized to obtain the set of relation edges to be stored.

[0046] Step 503: Write the set of entity nodes to be stored into the graph database, and create entity nodes corresponding to the set of entity nodes to be stored in the graph database. After the entity nodes are created, write the set of relation edges to be stored into the graph database. Based on the start node identifier and end node identifier of each set of relation edges in the relation edge set, create relation edges between the corresponding entity nodes that have been created. Use the graph database storage structure after the entity nodes and relation edges are created as a knowledge graph of the athlete's career archive. Specifically, this includes: selecting a graph database as the storage carrier. This graph database uses a graph structure to store data, supporting efficient storage and association query of nodes and edges. First, write the set of entity nodes to be stored into the graph database. The specific process is as follows: open the graph database's write interface, set the write rate to 100 to 200 records per second, and write each entity node in the set of entity nodes to be stored into the graph database one by one according to its storage format. In the graph database, create a knowledge graph for each entity node. Each node instance is created, and its attributes include a node identifier, type attribute, and description information, ensuring that each entity node can be accurately identified and queried in the graph database. After the entity nodes are created, the set of relation edges to be stored is written to the graph database. Specifically, the graph database continues to write each relation edge in the set of relation edges to be stored according to its storage format, maintaining the same write rate, through the graph database's write interface. For each relation edge, based on the start node identifier and end node identifier in its storage format, the corresponding two entity nodes are found in the entity nodes already created in the graph database, and a corresponding relation edge is created between these two entity nodes. The relation edge's attributes include a relation identifier and a relation type label. After all entity nodes and relation edges have been written to the graph database and created, the graph database forms a complete storage structure containing nodes and edges, which serves as the knowledge graph for the athlete's career archive.

[0047] This embodiment assigns unique identifiers to relationships, clarifies the start node, end node, and relationship type, standardizes the storage format of relationship edges, stores entities and relationships in a graph database to construct a knowledge graph, can intuitively present the relationships between various entities in an athlete's career record, integrates image and text modal data, and improves the utilization rate of athlete career record data.

[0048] like Figure 2 As shown, embodiments of the present invention also provide a system for constructing a career archive data map of a sports athlete, comprising: The acquisition module is used to acquire sports athlete career profile data containing image modal data and text modal data, convert the image modal data into visual semantic particle clusters, construct a feature projection manifold based on the visual semantic particle clusters, and delineate semantic sampling constraint polygons on the feature projection manifold. The computation module is used to calculate the shape complexity index of the semantic sampling constraint polygon. The shape complexity index is quantified by analyzing the curvature changes of the polygon boundary points and the ratio of area to perimeter. Based on the spatial distribution characteristics of visual semantic particle clusters, the semantic sampling constraint polygon is subdivided according to the shape complexity index to generate multiple visual feature partitions. The visual semantic particle clusters are mapped to the corresponding visual feature partitions, and the local feature entropy of the visual feature partitions is calculated. The local feature entropy and the shape complexity index are fused to generate cross-modal correction coefficients. The alignment module is used to input image modal data and text modal data into a pre-trained single- and dual-stream interaction model based on cross-modal correction coefficients, perform semantic similarity alignment, and optimize the alignment process using cross-modal correction coefficients to obtain a cross-modal entity set. The building module is used to construct a set of relationships between entities based on a cross-modal entity set, combined with predefined domain rules and a deep pre-trained language model; The output module is used to store entities as nodes and relationships as edges in a graph database to construct a knowledge graph of athletes' career profiles, based on the cross-modal entity set and the set of relationships between entities.

[0049] It should be noted that this system is a system corresponding to the above method. All implementation methods in the above method embodiments are applicable to this embodiment and can achieve the same technical effect.

[0050] Embodiments of the present invention also provide a computing device, including: a processor and a memory storing a computer program, wherein the computer program, when executed by the processor, performs the method described above. All implementations in the above method embodiments are applicable to this embodiment and can achieve the same technical effects.

[0051] Embodiments of the present invention also provide a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the method described above. All implementations in the above method embodiments are applicable to this embodiment and can achieve the same technical effects.

[0052] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for constructing a career archive data map of a sports athlete, characterized in that, The method includes: Step 1: Obtain athlete career profile data containing image modal data and text modal data; convert the image modal data into visual semantic particle clusters; construct a feature projection manifold based on the visual semantic particle clusters; and delineate semantic sampling constraint polygons on the feature projection manifold. Step 2: Calculate the shape complexity index of the semantic sampling constraint polygon. The shape complexity index is quantified by analyzing the curvature changes of the polygon boundary points and the ratio of area to perimeter. Based on the spatial distribution characteristics of the visual semantic particle clusters, the semantic sampling constraint polygon is subdivided according to the shape complexity index to generate multiple visual feature partitions. The visual semantic particle clusters are mapped to the corresponding visual feature partitions, and the local feature entropy of the visual feature partitions is calculated. The local feature entropy and the shape complexity index are fused to generate cross-modal correction coefficients. Step 3: Based on the cross-modal correction coefficient, input the image modal data and text modal data into the pre-trained single- and dual-stream interaction model, perform semantic similarity alignment, and optimize the alignment process using the cross-modal correction coefficient to obtain a cross-modal entity set; Step 4: Based on the cross-modal entity set, and combining predefined domain rules with a deep pre-trained language model, construct a set of relationships between entities; Step 5: Based on the cross-modal entity set and the set of relationships between entities, store the entities as nodes and the relationships as edges in the graph database to construct a knowledge graph of athletes' career profiles.

2. The method for constructing a sports athlete career record data map according to claim 1, characterized in that, Acquire athlete career profile data containing both image and text modal data; transform the image modal data into visual semantic particle clusters; construct a feature projection manifold based on the visual semantic particle clusters; and delineate semantic sampling constraint polygons on the feature projection manifold, including: Local feature detection is performed on image modal data to extract corner points, edge points, and color change points in the image, forming an image feature point set; Spatial analysis is performed on the image feature point set, and feature points that are adjacent in location and have similar feature attributes are merged into the same visual semantic particle cluster, resulting in multiple visual semantic particle clusters. Calculate the centroid of each visual semantic particle cluster, and construct a centroid point set in a high-dimensional space from all centroids; calculate the Euclidean distance between any two points in the centroid point set and construct a distance matrix; based on the distance matrix, map the high-dimensional centroid point set to a low-dimensional space, such that the difference between the Euclidean distance between each point in the low-dimensional space and the corresponding distance value in the distance matrix is ​​minimized, and obtain the coordinate point set in the low-dimensional space, and use the coordinate point set in the low-dimensional space as the feature projection manifold; In the low-dimensional space of the feature projection manifold, the convex hull of all low-dimensional coordinate points is calculated, and the polygonal region enclosed by the convex hull boundary is determined as the semantic sampling constraint polygon.

3. The method for constructing a sports athlete career record data map according to claim 2, characterized in that, The shape complexity index of semantically sampled constrained polygons is calculated. This index is quantified by analyzing the curvature changes at the polygon's boundary points and the ratio of area to perimeter, including: Extract the coordinates of all boundary points from the boundary of the semantically sampled constrained polygon, and sort the coordinate points in a clockwise direction to obtain the boundary point sequence; Traverse each boundary point in the boundary point sequence, calculate the local curvature change at the current point based on the direction of the line segment formed by the current point, the previous point, and the next point, and sum the local curvature changes of all boundary points to obtain the overall curvature change; calculate the area and perimeter of the semantic sampling constraint polygon to obtain the ratio of area to perimeter. The overall curvature change and the ratio of area to perimeter are normalized separately. The normalized overall curvature change and the normalized ratio of area to perimeter are then fused together, and the fused result is used as a shape complexity index.

4. The method for constructing a sports athlete career record data map according to claim 3, characterized in that, Based on the spatial distribution characteristics of visual semantic particle clusters, and combined with the shape complexity index, the semantic sampling constraint polygons are subdivided to generate multiple visual feature partitions. The visual semantic particle clusters are mapped onto the corresponding visual feature partitions, and the local feature entropy of the visual feature partitions is calculated. The local feature entropy and the shape complexity index are fused to generate cross-modal correction coefficients, including: Based on the spatial distribution density of visual semantic particle clusters within the semantic sampling constraint polygon, priority regions for subdivision are determined, and the subdivision granularity is determined based on the shape complexity index. The semantic sampling constraint polygon is recursively subdivided according to the priority regions and subdivision granularity. Each subdivision divides the current region into multiple sub-regions. During the recursive subdivision process, the number of visual semantic particle clusters contained in each sub-region is checked, the range between the number of visual semantic particle clusters in all sub-regions is calculated, and the range is compared with a preset uniformity threshold. The recursive subdivision stops when the range is less than the preset uniformity threshold, and all sub-regions obtained when the subdivision stops are used as visual feature partitions. Based on the spatial coordinates of each visual semantic particle cluster, each visual semantic particle cluster is assigned to the visual feature partition it falls into, thus obtaining the set of visual semantic particle clusters corresponding to each visual feature partition. For each visual feature partition, based on the set of visual semantic particle clusters corresponding to the visual feature partition, the total number of visual semantic particle clusters in the visual feature partition is counted, and the distribution probability of each visual semantic particle cluster in the visual feature partition is calculated. Based on the distribution probability, the information entropy of the visual feature partition is calculated, and the calculated information entropy is used as the local feature entropy of the visual feature partition. The local feature entropy corresponding to all visual feature partitions is summarized, the arithmetic mean of all local feature entropy is calculated, and the arithmetic mean is weighted with the shape complexity index to obtain the cross-modal correction coefficient.

5. The method for constructing a sports athlete career record data map according to claim 4, characterized in that, Based on cross-modal correction coefficients, image modal data and text modal data are input into a pre-trained single- and dual-stream interaction model for semantic similarity alignment. The alignment process is then optimized using cross-modal correction coefficients to obtain a cross-modal entity set, including: The attention weights in the pre-trained single- and dual-stream interaction model are redistributed according to the cross-modal correction coefficients, and the model obtained after redistributing the attention weights is used as the optimized single- and dual-stream interaction model. Image modal data is input into the image encoding branch of the optimized single- and dual-stream interaction model for global feature extraction to obtain image feature vectors; text modal data is input into the text encoding branch of the optimized single- and dual-stream interaction model for semantic feature extraction to obtain text feature vectors. Image feature vectors and text feature vectors are input into the interaction fusion layer of the optimized single-dual-stream interaction model. Cosine similarity is calculated on the image feature vectors and text feature vectors to obtain a semantic similarity score set. Image feature vectors and text feature vectors with similarity scores exceeding a preset matching threshold are extracted from the semantic similarity score set to generate candidate alignment pairs. Local regions are extracted from the image modal data corresponding to the candidate alignment pairs, and key phrases are extracted from the text modal data corresponding to the candidate alignment pairs. The local regions and key phrases are combined to obtain preliminary cross-modal entities. All preliminary cross-modal entities are aggregated and deduplicated to obtain a cross-modal entity set.

6. The method for constructing a career archive data map of a sports athlete according to claim 5, characterized in that, Based on the cross-modal entity set, and combined with predefined domain rules and a deep pre-trained language model, a set of relationships between entities is constructed, including: For each entity in the cross-modal entity set, type identification is performed. The entity category is determined based on the image modal data and text modal data corresponding to the entity, resulting in an entity set with category labels. Based on the entity set with category labels, rule matching conditions that match the entity category are extracted from a predefined domain rule base. The entity set with category labels is then input into the rule matching conditions for matching, resulting in a first relation candidate set. Entities containing text modal data are selected from the entity set with category labels. Based on the text modal data corresponding to the selected entities, text fragments containing at least two selected entities in the original text corpus are located. The located text fragments are used as context windows. The context windows are input into a deep pre-trained language model for analysis to identify the semantic relationships between entities in the text fragments and obtain a second relation candidate set. The first candidate set of relations and the second candidate set of relations are merged and deduplicated to obtain the deduplicated candidate set of relations. The confidence level of the candidate set of deduplicated relationships is evaluated, and the entity pairs and relationship type combinations with confidence levels exceeding the preset confidence threshold are selected. The selected combinations are taken as the set of relationships between entities.

7. The method for constructing a sports athlete career record data map according to claim 6, characterized in that, Based on the cross-modal entity set and the set of relationships between entities, entities are stored as nodes and relationships as edges in a graph database to construct a knowledge graph of athletes' career profiles, including: Assign a unique node identifier to each entity in the cross-modal entity set, determine the type attribute of the node based on the category label corresponding to the entity, extract the descriptive information of the node based on the image modal data and text modal data corresponding to the entity, and combine the node identifier, type attribute, and descriptive information into the storage format of the entity node to obtain the set of entity nodes to be stored. Assign a unique relation identifier to each entity pair and relation type combination in the set of relationships between entities. Based on the two entities in the entity pair, search for the corresponding node identifiers in the set of entity nodes to be stored. Use the two found node identifiers as the start node identifier and the end node identifier, respectively. Determine the relation type label based on the relation type corresponding to the entity pair. Combine the relation identifier, start node identifier, end node identifier, and relation type label into the storage format of the relation edge to obtain the set of relation edges to be stored. Write the set of entity nodes to be stored into the graph database, and create entity nodes corresponding to the set of entity nodes to be stored in the graph database. After the entity nodes are created, write the set of relation edges to be stored into the graph database. Based on the start node identifier and end node identifier of each set of relation edges, create relation edges between the corresponding entity nodes that have been created. The graph database storage structure after the entity nodes and relation edges are created is used as a knowledge graph of the athlete's career archive.

8. A system for constructing a career archive data map of a sports athlete, the system implementing the method as described in any one of claims 1 to 7, characterized in that, include: The acquisition module is used to acquire sports athlete career profile data containing image modal data and text modal data, convert the image modal data into visual semantic particle clusters, construct a feature projection manifold based on the visual semantic particle clusters, and delineate semantic sampling constraint polygons on the feature projection manifold. The calculation module is used to calculate the shape complexity index of semantically sampled constrained polygons. The shape complexity index is quantified by analyzing the curvature changes of the polygon boundary points and the ratio of area to perimeter. Based on the spatial distribution characteristics of visual semantic particle clusters, the semantic sampling constraint polygons are subdivided by combining the shape complexity index to generate multiple visual feature partitions; the visual semantic particle clusters are mapped to the corresponding visual feature partitions, and the local feature entropy of the visual feature partitions is calculated; the local feature entropy and the shape complexity index are fused to generate cross-modal correction coefficients. The alignment module is used to input image modal data and text modal data into a pre-trained single- and dual-stream interaction model based on cross-modal correction coefficients, perform semantic similarity alignment, and optimize the alignment process using cross-modal correction coefficients to obtain a cross-modal entity set. The building module is used to construct a set of relationships between entities based on a cross-modal entity set, combined with predefined domain rules and a deep pre-trained language model; The output module is used to store entities as nodes and relationships as edges in a graph database to construct a knowledge graph of athletes' career profiles, based on the cross-modal entity set and the set of relationships between entities.

9. A computing device, characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program that, when executed by a processor, implements the method as described in any one of claims 1 to 7.

Images