Data clustering optimization method based on large model
Patent Information
- Application Number
- CN202411402081.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-09
- Publication Date
- 2025-02-28
Smart Images
Figure CN119537599A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of data processing, and in particular relates to an optimization method for data clustering based on a large model. Background Art
[0002] Data is a fundamental strategic resource and revolutionary key element in the digital age. It is the core driving force for the development of the digital economy and the "reinforced concrete" for building the digital world. Digging deep into the "rich mine" of data and building digital productivity is of great significance and has broad prospects. However, we are currently facing imperfect data infrastructure, low data quality, lack of effective clustering data management, and security risks in the data processing process.
[0003] Data clustering is to cluster text data with similar topics together to form a category. The main purpose of data clustering technology is to explore the potential topic structure in text data, which is helpful for analyzing large-scale text data. The future development trends of data clustering technology include the following aspects: Improve clustering effect: Improve the accuracy and stability of topic clustering by optimizing clustering algorithms and models. Real-time data detection and tracking: Research methods for real-time detection and tracking of topics in order to quickly respond to social hot spots and emergencies. Cross-platform data clustering: Integrate text data from different platforms and different types to conduct cross-platform and cross-domain topic clustering analysis. Fusion of multimodal data: Combine multimodal data such as text, images, and audio to improve the effect and application value of topic clustering. Summary of the invention
[0004] The technical problem to be solved by the present invention is how to cluster text data of similar topics together. The present invention provides an optimization method for data clustering based on a large model. The center point of the first data and the center point of the second data are made close to each other through a line connection method, thereby realizing the possibility of the center point of the first data and the center point of the second data coinciding with each other. The stability of the coincidence of the center point of the first data and the center point of the second data is judged through the mass point coincidence method, thereby optimizing data clustering.
[0005] In order to achieve the above object, the present invention is implemented by the following technical solutions:
[0006] A method for optimizing data clustering based on a large model comprises the following steps:
[0007] Step 1): Obtain the first data and the second data, determine whether the subjects of the first data and the second data are similar, and find the center point of the first data and the center point of the second data respectively;
[0008] Step 2): connecting the center point of the first data and the center point of the second data; wherein the center point of the first data and the center point of the second data are connected by a line connection method;
[0009] Step 3): Let the center point of the first data coincide with the center point of the second data. After the center point of the first data coincides with the center point of the second data, use the mass point coincidence method to calculate the degree of coincidence between the center point of the first data and the center point of the second data.
[0010] Step 4): Cluster and merge the first data and the second data.
[0011] Optionally, in step 1), whether the subjects of the first data and the second data are similar is determined by using the following steps:
[0012] Step 11): Identify the part of speech of the first data and find out the keywords of the first data;
[0013] Step 12): Identify the part of speech of the second data and find out the keywords of the second data;
[0014] Step 13): Compare the meanings of the keywords of the first data and the keywords of the second data; if the meanings of the keywords of the first data and the keywords of the second data are the same or synonymous, the subjects of the first data and the second data are similar or the same; otherwise, the subjects of the first data and the second data are unrelated.
[0015] Optionally, in step 1), the center point of the first data or the center point of the second data is identified by using the following steps:
[0016] Step S1): taking the area where the same keyword appears most in the first data as the central area, and taking the geometric center of the central area as the center point of the first data;
[0017] Step S2): taking the area where the same keyword appears most in the second data as the central area, and taking the geometric center of the central area as the center point of the second data.
[0018] Optionally, in step 2), the center point of the first data and the center point of the second data are connected by a line connection method, which is the following steps:
[0019] Step 21): the center point of the first data moves in the plane, and the center point of the second data moves in the plane;
[0020] Step 22): connect the center point of the first data to the center point of the second data with a line, and connect the center point of the second data to the center point of the first data with a line;
[0021] Step 23): obtain two lines connecting the center point of the first data and the center point of the second data.
[0022] Optionally, in step 2), the line connection method is the following formula (1):
[0023] (1);
[0024] in, is the center point of the first data, The center point of the first data can move in the plane, To define the center point of the first data to move within the plane;
[0025] is the center point of the second data, The center point of the second data can be moved in the plane, To define the center point of the first data to move within the plane;
[0026] The center point of the first data Towards the center point of the second data The line connection, is the first tension level;
[0027] The center point of the second data Towards the center point of the first data The line connection, is the second tension level; = ,or ≠ ;
[0028] Represents the center point of the first data The center point of the second data It is connected by two lines.
[0029] Optionally, in step 3), the judgment of the coincidence process of the center point of the first data and the center point of the second data is performed in the following manner:
[0030] Step 31): within a unit time, the center point of the first data moves toward the center point of the second data;
[0031] Step 32): within a unit time, the center point of the second data moves toward the center point of the first data;
[0032] Step 33): When the distance between the center point of the first data and the center point of the second data is close to zero, and the two lines between the center point of the first data and the center point of the second data are close to zero, the center point of the first data coincides with the center point of the second data.
[0033] Furthermore, the particle coincidence method adopts the following formula (2):
[0034] (2);
[0035] in, is the center point of the first data shaking in space, The center point of the first data shaking in the space can be moved;
[0036] is the center point of the second data shaking in space, The center point of the second data swaying in the space can be moved;
[0037] The center point of the first data shaking in the space and the center point of the second data shaking in the space are close to each other;
[0038] is the coincidence judgment function of the data center points, It is used to determine whether the center point of the first data coincides with the center point of the second data, when the center point of the first data and the center point of the second data are close to each other and the center points of the two data coincide. The output value is 1, otherwise the center point of the first data and the center point of the second data are not close to each other and do not overlap. The output value is 0;
[0039] is the result of whether the center point of the first data coincides with the center point of the second data. Is 0 or 1.
[0040] Furthermore, the mass point coincidence method also includes the coincidence degree between the center point of the first data and the center point of the second data. The specific calculation of the coincidence degree is as follows:
[0041] (3);
[0042] After the center point of the first data coincides with the center point of the second data, is the distance between the center point of the first data and the center point of the second data in the t-1 second time period, is the distance between the center point of the first data and the center point of the second data in the t second time period; The change in distance between the center point of the first data and the center point of the second data in the time period t-1 seconds, which is equivalent to the time period t seconds; If it is equal to 1, the coincidence between the center point of the first data and the center point of the second data is stable; greater than 1, or If it is greater than 0 and less than 1, the coincidence between the center point of the first data and the center point of the second data is unstable; For the case of coincidence stability.
[0043] Optionally, in step 4), after the center point of the first data and the center point of the second data are stably overlapped, the first data and the second data are clustered and overlapped.
[0044] Beneficial effects of the present invention:
[0045] The present invention uses a line connection method to connect the center point of the first data and the center point of the second data after finding the center point of the first data and the center point of the second data respectively, and uses the mass point coincidence method to calculate the coincidence degree of the center point of the first data and the center point of the second data. The center point of the first data and the center point of the second data are brought close to each other by the line connection method, thereby realizing the possibility of the center point of the first data and the center point of the second data coinciding. The stability of the coincidence of the center point of the first data and the center point of the second data is judged by the mass point coincidence method, thereby optimizing data clustering. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0047] Figure 1 It is a work flow chart of the present invention;
[0048] Figure 2 It is the working flow chart of the line connection method of the present invention;
[0049] Figure 3 The figure is a workflow diagram for judging the coincidence of the center points of two data according to the present invention. DETAILED DESCRIPTION
[0050] The embodiments of the present application are described in detail below with reference to the accompanying drawings.
[0051] Example 1
[0052] like Figure 1 As shown, this embodiment provides an optimization method for data clustering based on a large model, comprising the following steps:
[0053] Step 1): Obtain the first data and the second data, determine whether the subjects of the first data and the second data are similar, and find the center point of the first data and the center point of the second data respectively;
[0054] Step 2): connecting the center point of the first data and the center point of the second data; wherein the center point of the first data and the center point of the second data are connected by a line connection method;
[0055] Step 3): Let the center point of the first data coincide with the center point of the second data. After the center point of the first data coincides with the center point of the second data, use the mass point coincidence method to calculate the degree of coincidence between the center point of the first data and the center point of the second data.
[0056] Step 4): Cluster and merge the first data and the second data.
[0057] Specifically, in step 1), it is determined whether the subjects of the first data and the second data are similar, using the following steps:
[0058] Step 11): Identify the part of speech of the first data and find out the keywords of the first data;
[0059] Step 12): Identify the part of speech of the second data and find out the keywords of the second data;
[0060] Step 13): Compare the meanings of the keywords of the first data and the keywords of the second data; if the meanings of the keywords of the first data and the keywords of the second data are the same or synonymous, the subjects of the first data and the second data are similar or the same; otherwise, the subjects of the first data and the second data are unrelated.
[0061] In step 1), the center point of the first data or the center point of the second data is identified, using the following steps:
[0062] Step S1): taking the area where the same keyword appears most in the first data as the central area, and taking the geometric center of the central area as the center point of the first data;
[0063] Step S2): The area where the same keyword appears most in the second data is taken as the central area, and the geometric center of the central area is taken as the center point of the second data.
[0064] In step 2), if Figure 2 As shown, the center point of the first data and the center point of the second data are connected by a line connection method, which is the following steps:
[0065] Step 21): the center point of the first data moves in the plane, and the center point of the second data moves in the plane;
[0066] Step 22): connect the center point of the first data to the center point of the second data with a line, and connect the center point of the second data to the center point of the first data with a line;
[0067] Step 23): obtain two lines connecting the center point of the first data and the center point of the second data.
[0068] In step 2), the line connection method is the following formula (1):
[0069] (1);
[0070] in, is the center point of the first data, The center point of the first data can move in the plane, To define the center point of the first data to move within the plane;
[0071] is the center point of the second data, The center point of the second data can be moved in the plane, To define the center point of the first data to move within the plane;
[0072] The center point of the first data Towards the center point of the second data The line connection, is the first tension level (unit: N / m);
[0073] The center point of the second data Towards the center point of the first data The line connection, is the second tension level (unit: N / m); = ,or ≠ ;
[0074] Represents the center point of the first data The center point of the second data It is connected by two lines.
[0075] It should be noted here that the first tension and the second tension It reflects the force per meter. The greater the force per meter, the greater the center point of the first data. Towards the center point of the second data The tighter the line connection, the center point of the second data Towards the center point of the first data The tighter the line connection, the closer the center point of the first data Towards the center point of the second data A line connects the center point of the second data Towards the center point of the first data The line connections are the respective line connections, so = ,or ≠ .
[0076] In step 3), if Figure 3 As shown, the judgment of the coincidence process of the center point of the first data and the center point of the second data is carried out in the following manner:
[0077] Step 31): within a unit time, the center point of the first data moves toward the center point of the second data;
[0078] Step 32): within a unit time, the center point of the second data moves toward the center point of the first data;
[0079] Step 33): When the distance between the center point of the first data and the center point of the second data is close to zero, and the two lines between the center point of the first data and the center point of the second data are close to zero, the center point of the first data coincides with the center point of the second data.
[0080] Furthermore, the particle coincidence method adopts the following formula (2):
[0081] (2);
[0082] in, is the center point of the first data shaking in space, The center point of the first data shaking in the space can be moved;
[0083] is the center point of the second data shaking in space, The center point of the second data swaying in the space can be moved;
[0084] The center point of the first data shaking in the space and the center point of the second data shaking in the space are close to each other;
[0085] is the coincidence judgment function of the data center points, It is used to determine whether the center point of the first data coincides with the center point of the second data, when the center point of the first data and the center point of the second data are close to each other and the center points of the two data coincide. The output value is 1, otherwise the center point of the first data and the center point of the second data are not close to each other and do not overlap. The output value is 0;
[0086] is the result of whether the center point of the first data coincides with the center point of the second data. Is 0 or 1.
[0087] Here, the center point of the first data and the center point of the second data will shake in space. The center point of the first data reflecting the shaking in the space is close to the center point of the second data reflecting the shaking in the space. After the center point of the first data coincides with the center point of the second data, the coincidence judgment function of the center points make The output value is 1, otherwise, the center point of the first data does not coincide with the center point of the second data. The coincidence judgment function of the center point make The output value is 0.
[0088] The mass point coincidence method also includes the coincidence degree between the center point of the first data and the center point of the second data. The specific calculation of the coincidence degree is as follows:
[0089] (3);
[0090] After the center point of the first data coincides with the center point of the second data, is the distance between the center point of the first data and the center point of the second data in the t-1 second time period, is the distance between the center point of the first data and the center point of the second data in the t second time period; The change in distance between the center point of the first data and the center point of the second data in the time period t-1 seconds, which is equivalent to the time period t seconds; If it is equal to 1, the coincidence between the center point of the first data and the center point of the second data is stable; greater than 1, or If it is greater than 0 and less than 1, the coincidence between the center point of the first data and the center point of the second data is unstable; For the case of coincidence stability.
[0091] Here, after the center point of the first data coincides with the center point of the second data, if the distance between the center point of the first data in the t-1 second time period and the distance between the center point of the second data in the t second time period do not change in the two time periods, then the coincidence between the center points of the two data is stable. Otherwise, if the distance between the center points of the two data changes in the two time periods, then the coincidence between the center points of the two data is unstable (specifically, the distance between the center points of the two data changes in the previous time period, but does not change in the next time period; or the distance between the center points of the two data does not change in the previous time period, but changes in the next time period).
[0092] In step 4), after the center point of the first data and the center point of the second data are stably overlapped, the first data and the second data are clustered and overlapped. Here, the first data covers the second data, or the second data covers the first data.
[0093] Example 2
[0094] Based on Example 1, the present invention mainly connects the center point of the first data with the center point of the second data. The center point of the first data and the center point of the second data are connected by two lines. The tension of the two lines can be the same or different, reflecting the different tightness of the two lines. When the two lines are close to each other, the tension of the two lines can be different, but when the center point of the first data and the center point of the second data are about to coincide, the tension of the two lines is made the same, so that the center point of the first data and the center point of the second data coincide.
[0095] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope recorded in the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.
Claims
1. A data clustering optimization method based on a large model, characterized in that: The steps include: Step 1): Obtain the first data and the second data, determine whether the subjects of the first data and the second data are similar, and find the center point of the first data and the center point of the second data respectively; Step 2): connecting the center point of the first data and the center point of the second data; wherein the center point of the first data and the center point of the second data are connected by a line connection method; Step 3): Let the center point of the first data coincide with the center point of the second data. After the center point of the first data coincides with the center point of the second data, use the mass point coincidence method to calculate the degree of coincidence between the center point of the first data and the center point of the second data. Step 4): Cluster and merge the first data and the second data.
2. The optimization method for data clustering based on a large model according to claim 1, characterized in that: In step 1), whether the subjects of the first data and the second data are similar is determined by using the following steps: Step 11): Identify the part of speech of the first data and find out the keywords of the first data; Step 12): Identify the part of speech of the second data and find out the keywords of the second data; Step 13): Compare the meanings of the keywords of the first data and the keywords of the second data; if the meanings of the keywords of the first data and the keywords of the second data are the same or synonymous, the subjects of the first data and the second data are similar or the same; otherwise, the subjects of the first data and the second data are unrelated.
3. The optimization method for data clustering based on a large model according to claim 1, characterized in that: In the step 1), the center point of the first data or the center point of the second data is identified by using the following steps: Step S1): taking the area where the same keyword appears most in the first data as the central area, and taking the geometric center of the central area as the center point of the first data; Step S2): taking the area where the same keyword appears most in the second data as the central area, and taking the geometric center of the central area as the center point of the second data.
4. The optimization method for data clustering based on a large model according to claim 1, characterized in that: In the step 2), the center point of the first data and the center point of the second data are connected by the line connection method, which is the following steps: Step 21): the center point of the first data moves in the plane, and the center point of the second data moves in the plane; Step 22): connect the center point of the first data to the center point of the second data with a line, and connect the center point of the second data to the center point of the first data with a line; Step 23): obtain two lines connecting the center point of the first data and the center point of the second data.
5. The optimization method for data clustering based on a large model according to claim 1, characterized in that: In the step 2), the line connection method is the following formula (1): (1)； in, is the center point of the first data, The center point of the first data can be moved in the plane, To define the center point of the first data to move within the plane; is the center point of the second data, The center point of the second data can be moved in the plane, To define the center point of the first data to move within the plane; The center point of the first data Towards the center point of the second data The line connection, is the first tension level; The center point of the second data Towards the center point of the first data The line connection, is the second tension level; = ,or ≠ ; Represents the center point of the first data The center point of the second data It is connected by two lines.
6. The optimization method for data clustering based on a large model according to claim 1, characterized in that: In step 3), the judgment of the coincidence process of the center point of the first data and the center point of the second data is performed in the following manner: Step 31): within a unit time, the center point of the first data moves toward the center point of the second data; Step 32): within a unit time, the center point of the second data moves toward the center point of the first data; Step 33): When the distance between the center point of the first data and the center point of the second data is close to zero, and the two lines between the center point of the first data and the center point of the second data are close to zero, the center point of the first data coincides with the center point of the second data.
7. The optimization method for data clustering based on a large model according to claim 6, characterized in that: The particle coincidence method adopts the following formula (2): (2)； in, is the center point of the first data shaking in space, The center point of the first data shaking in the space can be moved; is the center point of the second data shaking in space, The center point of the second data swaying in the space can be moved; The center point of the first data shaking in the space and the center point of the second data shaking in the space are close to each other; is the coincidence judgment function of the data center points, It is used to determine whether the center point of the first data coincides with the center point of the second data, when the center point of the first data and the center point of the second data are close to each other and the center points of the two data coincide. The output value is 1, otherwise the center point of the first data and the center point of the second data are not close to each other and do not overlap. The output value is 0; is the result of whether the center point of the first data coincides with the center point of the second data. Is 0 or 1.
8. The optimization method for data clustering based on a large model according to claim 7, characterized in that: The mass point coincidence method also includes the degree of coincidence between the center point of the first data and the center point of the second data. The specific calculation of the degree of coincidence is as follows: (3)； After the center point of the first data coincides with the center point of the second data, is the distance between the center point of the first data and the center point of the second data in the t-1 second time period, is the distance between the center point of the first data and the center point of the second data in the t second time period; The change in distance between the center point of the first data and the center point of the second data in the time period t-1 seconds, which is equivalent to the time period t seconds; If it is equal to 1, the coincidence between the center point of the first data and the center point of the second data is stable; greater than 1, or If it is greater than 0 and less than 1, the coincidence between the center point of the first data and the center point of the second data is unstable; For the case of coincidence stability.
9. The optimization method for data clustering based on a large model according to claim 1, characterized in that: In the step 4), after the center point of the first data and the center point of the second data are stably overlapped, the first data and the second data are clustered and overlapped.