Data processing method, electronic device, and medium

Abstract

This invention provides a data processing method, electronic device, and medium, relating to the fields of big data and deep learning. The specific implementation includes: acquiring multiple object data sets; determining that the number of object data sets is greater than a preset threshold; for any candidate object data set among the multiple object data sets, determining neighboring object data sets associated with the candidate object data set based on the candidate object data set's feature vector and a first distance threshold; in response to the existence of neighboring object data sets among the multiple object data sets, generating first candidate representation data sets based on the candidate object data sets and the neighboring object data sets; processing the multiple object data sets based on the candidate object data sets, the neighboring object data sets, and the first candidate representation data sets to obtain a first representation dataset; and determining that the number of first representation data sets contained in the first representation dataset is less than or equal to a preset threshold, and defining the first representation dataset as a first sample dataset.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, specifically to the fields of big data, artificial intelligence, deep learning, etc., and particularly to a data processing method and apparatus, a training method and apparatus for a target object detection model, a target object detection method, apparatus, electronic device, storage medium, and computer program product. Background Technology

[0002] In fields such as deep learning and machine learning, a large amount of sample data is typically required for model training. However, when the amount of sample data is too large, it not only affects the training efficiency of the model, but also leads to the accumulation of local noise, thereby affecting the accuracy and precision of the model. Summary of the Invention

[0003] This invention provides a data processing method and apparatus, a training method and apparatus for a target object detection model, a target object detection method, apparatus, electronic device, storage medium, and computer program product.

[0004] According to one aspect of the present invention, a data processing method is provided, comprising: acquiring multiple object data; determining that the number of the multiple object data is greater than a preset number threshold; for any candidate object data among the multiple object data, determining neighboring object data associated with the candidate object data based on the feature vector of the candidate object data and a first distance threshold; in response to the existence of neighboring object data among the multiple object data, generating first candidate representation data based on the candidate object data and the neighboring object data; wherein the first candidate representation data is used to replace the candidate object data and the neighboring object data; processing the multiple object data based on the candidate object data, the neighboring object data, and the first candidate representation data to obtain a first representation dataset; and determining that the number of first representation data contained in the first representation dataset is less than or equal to a preset number threshold, and determining the first representation dataset as a first sample dataset.

[0005] According to an embodiment of the present invention, processing multiple object data based on candidate object data, neighboring object data, and first candidate representation data to obtain a first representation dataset includes: deleting candidate object data and neighboring object data from the multiple object data to obtain a candidate object dataset; and determining the first representation dataset based on the candidate object dataset and the first candidate representation data.

[0006] According to an embodiment of the present invention, the first representation dataset includes a plurality of first representation data; in response to detecting that the candidate object dataset does not include object data, and determining that the number of the plurality of first representation data is greater than a preset number threshold, the data processing method further includes: determining the point density of each first representation data; determining the first representation data with the largest point density among the plurality of first representation data as candidate first representation data; determining the number of second candidate representation data associated with the candidate first representation data based on the point density of the candidate first representation data; performing representation data extraction on the plurality of first representation data based on the number of second candidate representation data to obtain a second representation dataset; and determining that the number of second representation data contained in the second representation dataset is less than or equal to a preset number threshold, and determining the second representation dataset as a second sample dataset.

[0007] According to an embodiment of the present invention, extracting representation data from multiple first representation data based on the number of second candidate representation data to obtain a second representation dataset includes: determining neighboring representation data associated with the candidate first representation data from the multiple first representation data according to the feature vector and a second distance threshold of the candidate first representation data; performing clustering processing on the candidate first representation data and the neighboring representation data based on the number of second candidate representation data to obtain a number of second candidate representation data; deleting the candidate first representation data and the neighboring representation data from the multiple first representation data to obtain a candidate representation dataset; and determining the second representation dataset based on the candidate representation dataset and the number of second candidate representation data.

[0008] According to an embodiment of the present invention, in response to determining that the number of second representation data contained in the second representation dataset is greater than a preset number threshold, the data processing method further includes: determining the affected first representation data associated with candidate first representation data among a plurality of first representation data; determining the point density of the affected first representation data and the point density of a number of second candidate representation data respectively; determining the second representation data with the largest point density in the second representation dataset as candidate second representation data based on the point density of the affected first representation data, the point density of the number of second candidate representation data, and the point density of other representation data in the candidate representation dataset besides the affected first representation data; and updating the point density of the candidate first representation data based on the point density of the candidate second representation data, and repeatedly performing the operation of determining whether the number of second representation data contained in the second representation dataset is less than or equal to the preset number threshold.

[0009] According to an embodiment of the present invention, determining the first representation data with the highest point density among a plurality of first representation data as candidate first representation data includes: sorting the plurality of first representation data according to the point density corresponding to each of the plurality of first representation data to obtain a representation data sequence; splitting the representation data sequence into a plurality of first representation data subsequences based on computing resources; wherein each first representation data subsequence includes a preset number of first representation data; for each first representation data subsequence, determining the first representation data with the highest point density among the preset number of first representation data as a first subsequence representation data; and determining the first subsequence representation data with the highest point density among the plurality of first subsequence representation data as candidate first representation data.

[0010] According to an embodiment of the present invention, in response to determining that the number of second representation data contained in the second representation dataset is greater than a preset number threshold, the data processing method further includes: determining the affected first representation data associated with candidate first representation data among a plurality of first representation data; storing the affected first representation data and a number of second candidate representation data into a representation data sequence to be sorted; for each first representation data subsequence, deleting representation data associated with candidate first representation data and neighboring representation data, as well as the affected first representation data, from the first representation data subsequence to obtain a second representation data subsequence; determining the point density of the affected first representation data and the point density of the number of second candidate representation data in the representation data sequence to be sorted, respectively; and determining the point density of the affected first representation data in the representation data sequence to be sorted. The point density of the number of second candidate representation data and the point density of representation data in multiple second representation data subsequences are determined. The affected first representation data and the number of second candidate representation data in the representation data sequence to be sorted are respectively assigned to multiple second representation data subsequences to obtain multiple third representation data subsequences. For each third representation data subsequence, the representation data with the highest point density in the third representation data subsequence is determined as the second subsequence representation data. The second subsequence representation data with the highest point density in the multiple second subsequence representation data is determined as the candidate second representation data. The point density of the candidate first representation data is updated based on the point density of the candidate second representation data. The operation of determining whether the number of second representation data contained in the second representation dataset is less than or equal to a preset number threshold is repeatedly performed.

[0011] According to an embodiment of the present invention, each object data includes an object data identifier; based on the number of second candidate representation data, representation data extraction is performed on multiple first representation data to obtain a second representation dataset, which further includes: for each second candidate representation data, determining candidate first representation data and neighboring representation data associated with the second candidate representation data; determining the object data identifier of the candidate first representation data and the object data identifier of the neighboring representation data according to the object data identifiers corresponding to the multiple object data; determining the object data identifier of the second candidate representation data according to the object data identifiers of the candidate first representation data and the object data identifiers of the neighboring representation data; and associating the object data identifier of the second candidate representation data with the second candidate representation data.

[0012] According to an embodiment of the present invention, determining the object data identifier of the second candidate representation data based on the object data identifier of the candidate first representation data and the object data identifier of the neighboring representation data includes: determining the number of object data identifiers belonging to the same data identifier category among the object data identifiers of the candidate first representation data and the object data identifiers of the neighboring representation data; and taking the object data identifier with the largest number as the object data identifier of the second candidate representation data.

[0013] According to an embodiment of the present invention, the number of second candidate characterization data is positively correlated with the point density of the first candidate characterization data.

[0014] According to an embodiment of the present invention, determining the point density of each first representation data includes: for each first representation data, determining the number of neighboring representation data associated with the first representation data from a plurality of first representation data based on a second distance threshold and the feature vector of the first representation data; and determining the number of neighboring representation data as the point density.

[0015] According to an embodiment of the present invention, the data processing method further includes: in response to determining that there is no neighboring object data associated with the candidate object data among a plurality of object data, determining the candidate object data as first candidate representation data.

[0016] According to an embodiment of the present invention, each object data includes an object data identifier; generating first candidate representation data based on candidate object data and neighboring object data further includes: determining the object data identifier of the candidate object data and the object data identifier of the neighboring object data based on the object data identifiers corresponding to each of the multiple object data; determining the object data identifier of the first candidate representation data based on the object data identifier of the candidate object data and the object data identifier of the neighboring object data; and associating the object data identifier of the first candidate representation data with the first candidate representation data.

[0017] According to an embodiment of the present invention, determining the object data identifier of the first candidate representation data based on the object data identifier of the candidate object data and the object data identifier of the neighboring object data includes: determining the number of object data identifiers belonging to the same data identifier category among the object data identifiers of the candidate object data and the object data identifiers of the neighboring object data; and taking the object data identifier with the largest number as the object data identifier of the first candidate representation data.

[0018] According to another aspect of the present invention, a method for training a target object detection model is provided, comprising: acquiring a sample dataset; iteratively training a deep learning model using the sample dataset until the output of the deep learning model satisfies the iteration stopping condition or the cumulative number of iterations reaches a preset threshold, thereby obtaining a target object detection model; wherein the sample dataset is obtained using the data processing method described above.

[0019] According to another aspect of the present invention, a target object detection method is provided, comprising: inputting data to be processed into a target object detection model to obtain a detection result for the data to be processed; wherein the target object detection model is trained using the target object detection model training method described above.

[0020] According to another aspect of the present invention, a data processing apparatus is provided, comprising: a first acquisition module for acquiring a plurality of object data; a first determination module for determining that the number of the plurality of object data is greater than a preset number threshold, and for any candidate object data among the plurality of object data, determining neighboring object data associated with the candidate object data based on the feature vector of the candidate object data and a first distance threshold; a generation module for generating first candidate representation data based on the candidate object data and the neighboring object data in response to the existence of neighboring object data among the plurality of object data; wherein the first candidate representation data is used to replace the candidate object data and the neighboring object data; a processing module for processing the plurality of object data based on the candidate object data, the neighboring object data and the first candidate representation data to obtain a first representation dataset; and a second determination module for determining that the number of first representation data contained in the first representation dataset is less than or equal to a preset number threshold, and determining the first representation dataset as a first sample dataset.

[0021] According to another aspect of the present invention, a training apparatus for a target object detection model is provided, comprising: a second acquisition module for acquiring a sample dataset; and a training module for iteratively training a deep learning model using the sample dataset until the output of the deep learning model satisfies the iteration stopping condition or the cumulative number of iterations reaches a preset threshold, thereby obtaining a target object detection model; wherein the sample dataset is obtained using the data processing apparatus described above.

[0022] According to another aspect of the present invention, a target object detection apparatus is provided, comprising: an input module for inputting data to be processed into a target object detection model to obtain a detection result for the data to be processed; wherein the target object detection model is trained using the training apparatus for the target object detection model described above.

[0023] According to another aspect of the present invention, an electronic device is provided, comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors perform the method described above.

[0024] According to another aspect of the present invention, a computer-readable storage medium is provided having executable instructions stored thereon, which, when executed by a processor, cause the processor to perform the method described above.

[0025] According to another aspect of the present invention, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the method described above. Attached Figure Description

[0026] To further illustrate the technical content of this invention, the following will describe it in detail with reference to examples and accompanying drawings, wherein:

[0027] Figure 1 This is an exemplary system architecture diagram of the data processing method and apparatus, the target object detection model training method and apparatus, and the target object detection method and apparatus that can be applied according to embodiments of the present invention.

[0028] Figure 2 This is a flowchart of a data processing method according to an embodiment of the present invention;

[0029] Figure 3 This is a schematic diagram of the process of generating the first sample dataset according to an embodiment of the present invention;

[0030] Figure 4 This is a schematic diagram of the process of generating a second representation dataset according to an embodiment of the present invention;

[0031] Figure 5 This is a schematic diagram illustrating the reordering of point densities of each second representation data in the second representation dataset according to an embodiment of the present invention.

[0032] Figure 6A This is a schematic diagram illustrating the effect of data classification using relevant technologies;

[0033] Figure 6B This is a schematic diagram illustrating the effect of data classification based on the technical solution of this invention.

[0034] Figure 7 This is a flowchart of a training method for a target object detection model according to an embodiment of the present invention;

[0035] Figure 8 This is a flowchart of a target object detection method according to an embodiment of the present invention;

[0036] Figure 9 This is a block diagram of a data processing apparatus according to an embodiment of the present invention;

[0037] Figure 10 This is a block diagram of a training apparatus for a target object detection model according to an embodiment of the present invention;

[0038] Figure 11 This is a block diagram of a target object detection device according to an embodiment of the present invention; and

[0039] Figure 12 This is a block diagram of an electronic device used to implement the data processing method, the target object detection model training method, and the target object detection method in the embodiments of the present invention. Detailed Implementation

[0040] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the embodiments and the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0041] Figure 1 This is a schematic diagram of an exemplary system architecture for applying data processing methods and apparatus, target object detection model training methods and apparatus, and target object detection methods and apparatus according to embodiments of the present invention. It should be noted that... Figure 1 The examples shown are merely examples of system architectures that can be applied to embodiments of the present invention, in order to help those skilled in the art understand the technical content of the present invention, but do not mean that embodiments of the present invention cannot be used in other devices, systems, environments or scenarios.

[0042] like Figure 1 As shown, the system architecture 100 according to this embodiment may include terminal devices 101, 102, and 103, a network 104, and a server 105. The network 104 serves as a medium for providing a communication link between the terminal devices 101, 102, and 103 and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0043] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103. Examples include object detection applications, web browser applications, search applications, instant messaging tools, email clients, or social media platform software (for example only).

[0044] Terminal devices 101, 102, and 103 can be various electronic devices with displays and web browsing capabilities, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0045] Server 105 can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud computing, network services, and middleware services.

[0046] Server 105 can be a server that provides various services, such as a backend management server that supports websites browsed by users using terminal devices 101, 102, and 103 (for example only). The backend management server can analyze and process data such as received user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0047] For example, server 105 can obtain multiple object data from terminal devices 101, 102, and 103 via network 104. When the number of multiple object data exceeds a preset threshold, server 105 can process the multiple object data to generate a sample dataset. The sample data contained in the sample dataset can replace the multiple object data.

[0048] In some embodiments, server 105 can also train the deep learning model using the aforementioned sample dataset. After training the deep learning model, server 105 can also use the trained deep learning model (e.g., an object detection model) to detect target objects in the data to be processed. In some examples, server 105 can also send the trained deep learning model (e.g., an object detection model) to terminal devices 101, 102, and 103. In this way, users can apply the object detection model in the terminal devices to perform object detection.

[0049] It should be noted that the data processing method provided in the embodiments of the present invention can generally be executed by server 105. Correspondingly, the data processing device provided in the embodiments of the present invention can generally be located in server 105. The data processing method provided in the embodiments of the present invention can also be executed by a server or server cluster that is different from server 105 and capable of communicating with terminal devices 101, 102, 103 and / or server 105. Correspondingly, the data processing device provided in the embodiments of the present invention can also be located in a server or server cluster that is different from server 105 and capable of communicating with terminal devices 101, 102, 103 and / or server 105.

[0050] It should be noted that the training method for the target object detection model provided in this embodiment of the invention can generally be executed by server 105. Correspondingly, the training device for the target object detection model provided in this embodiment of the invention can generally be located in server 105. The training method for the target object detection model provided in this embodiment of the invention can also be executed by a server or server cluster that is different from server 105 and capable of communicating with terminal devices 101, 102, 103 and / or server 105. Correspondingly, the training device for the target object detection model provided in this embodiment of the invention can also be located in a server or server cluster that is different from server 105 and capable of communicating with terminal devices 101, 102, 103 and / or server 105.

[0051] It should be noted that the target object detection method provided in the embodiments of the present invention can generally be executed by server 105. Correspondingly, the target object detection device provided in the embodiments of the present invention can generally be located in server 105. The target object detection method provided in the embodiments of the present invention can also be executed by a server or server cluster that is different from server 105 and capable of communicating with terminal devices 101, 102, 103 and / or server 105. Correspondingly, the target object detection device provided in the embodiments of the present invention can also be located in a server or server cluster that is different from server 105 and capable of communicating with terminal devices 101, 102, 103 and / or server 105.

[0052] Alternatively, the target object detection method provided in the embodiments of the present invention can also be executed by terminal devices 101, 102, or 103. Correspondingly, the target object detection device provided in the embodiments of the present invention can also be disposed in terminal devices 101, 102, or 103.

[0053] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0054] It should be noted that the sequence numbers of the operations in the following methods are for descriptive purposes only and should not be considered as indicating the execution order of the operations. Unless explicitly stated otherwise, the method does not need to be executed in the exact order shown.

[0055] Figure 2 This is a flowchart of a data processing method according to an embodiment of the present invention.

[0056] like Figure 2 As shown, the data processing method 200 includes operations S210 to S250.

[0057] In operation S210, data from multiple objects is retrieved.

[0058] In operation S220, it is determined that the number of multiple object data is greater than a preset number threshold. For any candidate object data among the multiple object data, based on the feature vector of the candidate object data and a first distance threshold, neighboring object data associated with the candidate object data is determined from the multiple object data.

[0059] In operation S230, in response to the existence of neighboring object data among multiple object data, a first candidate representation data is generated based on the candidate object data and the neighboring object data.

[0060] In operation S240, multiple object data are processed based on candidate object data, neighboring object data, and first candidate representation data to obtain the first representation dataset.

[0061] In operation S250, it is determined that the number of first representation data contained in the first representation dataset is less than or equal to a preset number threshold, and the first representation dataset is determined as the first sample dataset.

[0062] According to embodiments of the present invention, the object data includes, but is not limited to, data of the types including text, images, videos, audio, or location information carrying identification data (wherein the identification data is associated with the location information), etc., and the specific type can be selected according to actual needs, without limitation. In one example, the location information carrying identification data may include latitude and longitude information carrying administrative region code information, for example. Of course, the present invention is not limited to this.

[0063] Multiple object data sets can be used to construct sample datasets for model training. Of course, the uses of multiple object data sets are not limited to this. For simplicity, the following example illustrates how multiple object data sets can be used to construct sample datasets.

[0064] According to an embodiment of the present invention, a preset quantity threshold is used to indicate the expected amount of sample data in the sample dataset to be generated. The preset quantity threshold can be set according to actual conditions, and the present invention does not limit it.

[0065] In this embodiment of the invention, the number of multiple object data can be compared with a preset quantity threshold to determine whether further processing of the multiple object data is required to obtain a sample dataset with the desired data volume.

[0066] For example, if the number of multiple object data sets is determined to be greater than a preset threshold, meaning the number of multiple object data sets exceeds the expected data volume, it indicates that there is redundant object data among them. In this case, the multiple object data sets need to be processed to obtain a sample dataset with the expected data volume. Conversely, if the number of multiple object data sets is determined to be less than or equal to the preset threshold, it indicates that the number of multiple object data sets is appropriate. In this case, no processing is required, and the sample dataset can be obtained directly from the multiple object data sets.

[0067] In an embodiment of the present invention, when the number of multiple object data is determined to be greater than a preset number threshold, for any candidate object data among the multiple object data, based on the feature vector of the candidate object data and a first distance threshold, it is determined whether there is neighboring object data associated with the candidate object data among the multiple object data. Neighboring object data refers to object data among the multiple object data that has similar characteristics to the candidate object data.

[0068] It's understandable that for any two sets of object data, the more similar their features, the closer they will be; conversely, the less similar they are, the closer they will be to each other. Therefore, the distance between object data can be used to determine neighboring object data associated with candidate object data.

[0069] For example, for any candidate object data, the distance between the feature vector of the candidate object data and the feature vector of another candidate object data is determined. The other candidate object data refers to any object data among multiple object data that is distinct from the first candidate object data. If the distance between the feature vector of the first candidate object data and the feature vector of the other candidate object data is less than or equal to a first distance threshold, then the other candidate object data is identified as a neighboring object data associated with the first candidate object data. Conversely, if the distance between the feature vector of the first candidate object data and the feature vector of the other candidate object data is greater than the first distance threshold, then the other candidate object data is considered not a neighboring object data associated with the first candidate object data. Based on the above method, it is possible to determine whether there is neighboring object data associated with a candidate object data among multiple object data, and the number of neighboring object data associated with the candidate object data.

[0070] It should be noted that the first distance threshold described above can be one or more preset distance values, or one or more preset distance ranges. The specific value can be set according to the actual situation, and there is no limitation here.

[0071] In addition, when determining the distance between the feature vector of a candidate object data and the feature vector of another candidate object data, any one or more distance calculation methods in related technologies can be used, such as including but not limited to Euclidean distance, Mahalanobis distance, Manhattan distance, etc., and this invention does not limit it.

[0072] In this embodiment of the invention, if it is determined that there is at least one neighboring object data associated with the candidate object data among multiple object data, a first candidate representation data can be generated based on the candidate object data and the at least one neighboring object data.

[0073] Since each neighboring object data point shares similar characteristics with the candidate object data, at least one neighboring object data point and the candidate object data can be considered as object data of the same class. Thus, a first candidate representation data point can be generated within the scope containing the candidate object data and at least one neighboring object data point. This first candidate representation data point is used to replace the candidate object data and at least one neighboring object data point. In other words, the first candidate representation data point can be used to represent the candidate object data and at least one neighboring object data point, which is beneficial for subsequent processing of multiple object data points.

[0074] In one example, a cluster center can be generated based on a clustering algorithm, using candidate object data and at least one neighboring object data, and this cluster center can be designated as the first candidate representation data. In this embodiment of the invention, a suitable clustering algorithm can be selected to generate the cluster center according to actual needs; no limitation is made here.

[0075] Next, based on the candidate object data, at least one neighboring object data associated with the candidate object data, and the first candidate representation data, the multiple object data are processed to obtain the first representation dataset.

[0076] In this embodiment of the invention, since the first candidate representation data can represent candidate object data and at least one neighboring object data, when processing multiple object data, the first candidate representation data can be used to replace the candidate object data and at least one neighboring object data in the multiple object data to obtain a first representation dataset. The first representation dataset includes multiple first representation data, which can represent multiple object data. Processing multiple object data in the above manner can simplify the processing of redundant data in the multiple object data, thereby facilitating the acquisition of sample data of the desired data volume.

[0077] Next, it is determined whether the number of first representation data in the first representation dataset is less than or equal to a preset quantity threshold, that is, whether the number of first representation data is less than or equal to the expected data volume. If it is determined that the number of first representation data in the first representation dataset is less than or equal to the preset quantity threshold, then the first representation dataset is determined as the first sample dataset.

[0078] If it is determined that the number of first representation data contained in the first representation dataset is greater than a preset number threshold, another candidate object data can be determined from multiple object data, and the other candidate object data can be used to update the above candidate object data. The operation of determining that the number of first representation data contained in the first representation dataset is less than or equal to the preset number threshold is repeated until it is determined that the number of first representation data contained in the first representation dataset reaches the preset number threshold; or after processing all object data, if it is determined that the number of first representation data contained in each of the obtained first representation datasets has not reached the preset number threshold, the above operations S220 to S250 are stopped.

[0079] According to an embodiment of the present invention, when it is determined that the number of multiple object data exceeds the desired data volume, first candidate representation data is generated by utilizing candidate object data and neighboring object data associated with the candidate object data. Then, the first candidate representation data is used to replace the candidate object data and neighboring object data in the multiple object data to obtain a first sample dataset. This enables simplified processing of redundant data in the multiple object data, thereby obtaining a first sample dataset with the desired data volume.

[0080] Figure 3 This is a schematic diagram illustrating the process of generating the first sample dataset according to an embodiment of the present invention. (Refer to the following...) Figure 3 The process of generating the first sample dataset is illustrated below. In this embodiment of the invention, each data point can be represented as a data point. For example, if the object data is video, image, or audio, the object data can be input into a trained neural network (e.g., ResNet, VGG, Transformer, etc., which can be used to process images; and correspondingly, BLSTM-RNN, WaveNet, GPT, etc., which can be used to process audio) to extract features and obtain the corresponding feature vector. The feature vector can be represented using data points or other forms (e.g., lines, surfaces, etc.). It should be understood that the above representation is only for understanding the technical solution described in the embodiments of the present invention, but the solution of the present invention is not limited thereto.

[0081] like Figure 3 As shown in 310, multiple object data include, for example, object data P. a Object data P bObject data P c Object data P d Object data P e Object data P f Object data P g Object data P h Object data P j Object data P k Object data P m .

[0082] When the number of multiple object data sets exceeds a preset threshold, for any candidate object data set among the multiple object data sets, for example, object data P can be selected. d It is determined to be a candidate object data, based on the object data P d Using the feature vector and a first distance threshold, neighboring object data associated with a candidate object data can be determined from multiple object data sets. For example, object data P can be used as a reference. d Centered on the first distance threshold, all object data within a radius are determined as the candidate object data (i.e., object data P). d Related neighboring object data, i.e., object data P a Object data P b Object data P c Determined to be related to object data P d Related neighboring object data.

[0083] Then, based on the candidate object data and all neighboring object data, the first candidate representation data is generated. For example... Figure 3 As shown in 320, for example, object data P can be... a Object data P b Object data P c Object data P d Clustering is performed to generate a cluster center, which is then used as the first candidate representation data (i.e., ct1 shown in 320). The first candidate representation data ct1 can be used to replace the object data P. a Object data P b Object data P c and object data P d .

[0084] Next, candidate object data and neighboring object data are removed from multiple object datasets to obtain a candidate object dataset. Then, based on the candidate object dataset and the first candidate representation data, the first representation dataset is determined.

[0085] like Figure 3 As shown in 320 and 330, for example, deleting object data P from multiple object data.b and object data P d Related neighboring object data (i.e., object data P) a Object data P b and object data P c First, a candidate object dataset is obtained. Then, based on the candidate object dataset and the first candidate representation data ct1, the first representation dataset is obtained. The first representation dataset includes the first candidate representation data ct1 and object data P. e Object data P f Object data P g Object data P h Object data P j Object data P k Object data P m The individual data points contained in the first representation dataset constitute the first representation data.

[0086] Next, it can be determined whether the number of first representation data in the first representation dataset is less than or equal to a preset number threshold. If so, the first representation dataset is determined as the first sample dataset. If not, another candidate object data is determined from multiple object data, and the process described above is used to process the other candidate object data to obtain another first representation dataset.

[0087] Assuming the number of first representation data points in the aforementioned first representation dataset exceeds a preset threshold, such as... Figure 3 As shown in 330, object data P can also be... g (For example only) another candidate object data is identified, and based on that object data P g Using the feature vectors and a first distance threshold, we determine the object data (i.e., object data P) from multiple object data sets that is related to another candidate object data set. g Related neighboring object data. For example, related to object data P. g Related neighboring object data includes object data P f and object data P h .

[0088] Next, based on the data of another candidate object and all neighboring object data, another first candidate representation data is generated. For example... Figure 3 As shown in 330, for example, object data P can be... g Object data P f and object data P h Clustering is performed to generate another cluster center, which is then used as another first candidate representation data (i.e., ct2 shown in 330). The first candidate representation data ct2 can be used to replace the object data P. gObject data P f and object data P h .

[0089] Next, delete another candidate object data and its corresponding neighboring object data from the multiple object data to obtain another candidate object dataset. Then, based on the other candidate object dataset and another first candidate representation data, determine another first representation dataset.

[0090] like Figure 3 As shown in 330 and 340, for example, deleting object data P from multiple object data. g and object data P g Related neighboring object data (i.e., object data P) f and object data P h This process yields another candidate object dataset. Then, based on this other candidate object dataset and the first candidate representation data ct2, another first representation dataset is obtained. This other first representation dataset includes the first candidate representation data ct1, the first candidate representation data ct2, and object data P. e Object data P j Object data P k Object data P m The individual data points contained in another first representation dataset constitute the updated first representation data.

[0091] Next, it can be determined whether the number of updated first representation data is less than or equal to a preset number threshold. If so, another first representation dataset is determined as the first sample dataset. If not, another candidate object data is determined from multiple object data, and the process described above is used to process the other candidate object data to obtain another first representation dataset.

[0092] Assuming the number of the first representation data updated above is still greater than the preset threshold, it can also be obtained from the remaining object data (including object data P). e Object data P j Object data P k Object data P m Another candidate object data can be identified from the object data P. e This is identified as another candidate object data. Afterwards, it can be determined based on object data P. e Using the feature vectors and a first distance threshold, we determine the object data (i.e., object data P) from multiple object data sets that is related to another candidate object data set. e Related neighboring object data. For example... Figure 3 As shown in 340, with object data P eCentered on the first distance threshold, the non-existent data P of the object is considered. e Related neighboring object data. At this point, another candidate object data (i.e., object data P) can be considered. e It is determined to be another first candidate representation data, denoted as first candidate representation data ct3.

[0093] Next, delete another candidate object data (i.e., object data P) from among multiple object data. e This process yields another candidate object dataset. Then, based on this other candidate object dataset and the first candidate representation data ct3, another first representation dataset is determined. This other first representation dataset includes first candidate representation data ct1, first candidate representation data ct2, first candidate representation data ct3, and object data P. j Object data P k Object data P m The individual data points contained in another first representation dataset constitute the updated first representation data.

[0094] Next, determine whether the number of updated first representation data contained in the other first representation dataset is less than or equal to a preset number threshold. If yes, then the other first representation dataset is determined as the first sample dataset. If no, then the remaining object data (including object data P) is used. j Object data P k Object data P m In the first candidate object dataset, another candidate object data is determined, and the process described above is used to continue processing the other candidate object data; this operation is repeated until the number of first characterization data contained in the first characterization dataset reaches a preset threshold, or when it is detected that the other candidate object dataset does not contain object data, the above operation is stopped.

[0095] In this embodiment of the invention, when the number of multiple object data exceeds the desired data volume, first candidate representation data is generated by utilizing each candidate object data and its corresponding neighboring object data. Then, the first candidate representation data is used to replace the candidate object data and neighboring object data in the multiple object data to obtain a first sample dataset. This simplifies the processing of redundant data in the multiple object data, thereby obtaining a first sample dataset with the desired data volume.

[0096] According to embodiments of the present invention, each of the above-described object data may include an object data identifier. The object data identifier is used to indicate descriptive information associated with the object data. The descriptive information includes, but is not limited to, the category to which the object data belongs, the target object contained in the object data and the category to which the target object belongs, or other information related to the object data, etc.

[0097] In the process of generating the first candidate representation data, the object data identifier corresponding to the first candidate representation data can also be determined based on the object data identifier of each object data.

[0098] In one example, when it is determined that there are neighboring object data associated with candidate object data among multiple object data, the object data identifier of the candidate object data and the object data identifier of the neighboring object data can be determined according to the object data identifiers corresponding to each of the multiple object data.

[0099] Next, based on the object data identifiers of the candidate object data and the object data identifiers of neighboring object data, the object data identifier of the first candidate representation data is determined. For example, the number of object data identifiers belonging to the same data identifier category among the object data identifiers of the candidate object data and the object data identifiers of neighboring object data can be determined. Then, the object data identifier with the largest number is selected as the object data identifier of the first candidate representation data.

[0100] For example, candidate object data P d The object data identifier is object data identifier 1. This is related to the candidate object data P. d Related neighboring object data includes object data P a Object data P b and object data P c Object data P a Object data P b and object data P c The corresponding object data identifiers are object data identifier 1, object data identifier 2, and object data identifier 1, respectively. The candidate object data P... d The object data identifiers of the objects and the object data identifiers of the neighboring object data are aggregated to obtain one object data identifier 2 and three object data identifiers 1. The object data identifier with the largest number, i.e., object data identifier 1, is selected as the object data identifier of the first candidate representation data.

[0101] Next, the object data identifier of the first candidate representation data is associated with the first candidate representation data. For example, the first candidate representation data can be associated with object data identifier 1 and stored thereafter, and object data identifier 1 can be used to describe the relevant information contained in the first candidate representation data.

[0102] In this embodiment of the invention, the object data identifier of the first candidate representation data is determined by using the object data identifier of the candidate object data and the object data identifier of the neighboring object data. In this way, when the first candidate representation data is used to replace the candidate object data and the neighboring object data, the object data identifier of the first candidate representation data can also be used to represent the object data identifier situation corresponding to the candidate object data and the neighboring object data.

[0103] In another example, for instance, when it is determined that there is no neighboring object data associated with candidate object data among multiple object data, the object data identifier of the candidate object data can be determined as the object data identifier of the first candidate representation data.

[0104] For example, candidate object data P e The object data identifier is object data identifier 3. If it is determined that there is no candidate object data P among multiple object data, then... e Related neighboring object data can be used to identify candidate object data P. e The object data identifier 3 is determined as the first candidate representation data and is also determined as the object data identifier of the first candidate representation data.

[0105] In some embodiments, if it is detected that the candidate object dataset does not contain object data, and it is determined that the number of first representation data contained in the first representation dataset is greater than a preset threshold, multiple first representation data in the first representation dataset can be processed based on a clustering method to obtain a second sample dataset with the desired data volume. This will be illustrated below with reference to specific embodiments.

[0106] First, when it is detected that the candidate object dataset does not include object data, and it is determined that the number of first representation data contained in the first representation dataset is greater than a preset number threshold, the point density of each first representation data can be determined.

[0107] For example, for each first representation data, based on a second distance threshold and the feature vector of the first representation data, the number of neighboring representation data associated with the first representation data is determined from a plurality of first representation data. Then, the number of neighboring representation data is determined as the point density of the first representation data.

[0108] In this embodiment of the invention, neighboring representation data is used to indicate first representation data in the first representation data set that has similar features to the first representation data. The process of determining neighboring representation data associated with the first representation data based on a second distance threshold and the feature vector of the first representation data is similar to the process of determining neighboring object data described above, and will not be repeated here.

[0109] It should be noted that the second distance threshold is different from the first distance threshold. The second distance threshold can be one or more preset distance values, or one or more preset distance ranges. The specific value can be set according to the actual situation, and there is no limitation here.

[0110] Next, the first representation data with the highest point density among multiple first representation data is determined as the candidate first representation data.

[0111] For example, after determining the point density corresponding to each of the multiple first representation data, the point densities of each of the multiple first representation data are sorted. Then, based on the point density sorting results, the first representation data with the highest point density among the multiple first representation data is determined as the candidate first representation data.

[0112] Next, based on the point density of the first candidate representation data, the number of second candidate representation data associated with the first candidate representation data is determined.

[0113] In this embodiment of the invention, the second candidate representation data can be used to replace the first candidate representation data and the neighboring representation data associated with the first candidate representation data. Subsequently, the second candidate representation data can be used to replace the first candidate representation data and the corresponding neighboring representation data to simplify the processing of redundant data in the multiple first representation data sets, thereby obtaining a second sample dataset with the desired data volume.

[0114] In order to enable the second candidate representation data to better replace the first candidate representation data and the corresponding neighboring representation data, in this embodiment of the invention, the number of second candidate representation data associated with the first candidate representation data can be determined based on the point density of the first candidate representation data.

[0115] It's understandable that a higher point density for a primary representation data point means a larger number of neighboring representation data points associated with it. Consequently, more secondary candidate representation data points are needed to represent the candidate primary representation data point and its corresponding neighboring representation data. Conversely, a lower point density for a primary representation data point means a smaller number of neighboring representation data points associated with it. Consequently, fewer secondary candidate representation data points can be used to represent the candidate primary representation data point and its corresponding neighboring representation data. In other words, the number of secondary candidate representation data points is positively correlated with the point density of the candidate primary representation data point.

[0116] In one example, the number of second candidate representation data can be determined using the following formula (1).

[0117]

[0118] In formula (1), N cluster N represents the number of second candidate representation data.density N represents the point density of the candidate first representation data. e N represents the expected amount of sample data. retain This indicates the number of multiple first-characterization data.

[0119] Next, based on the number of second candidate representation data, representation data is extracted from multiple first representation data to obtain the second representation dataset.

[0120] For example, based on the feature vector of the candidate first representation data and a second distance threshold, neighboring representation data associated with the candidate first representation data can be determined from multiple first representation data. This process is similar to the process of determining neighboring object data described above, and will not be repeated here.

[0121] Subsequently, based on the number of second candidate representation data, the candidate first representation data and the neighboring representation data are clustered to obtain a number of second candidate representation data. For example, clustering algorithms from related technologies, such as, but not limited to, the k-means clustering algorithm, can be used to cluster the candidate first representation data and the neighboring representation data to obtain a number of second candidate representation data.

[0122] Next, a number of second candidate representation data are used to replace the candidate first representation data and their corresponding neighboring representation data in the multiple first representation data, thereby simplifying the processing of redundant data in the multiple first representation data. For example, candidate first representation data and neighboring representation data can be deleted from the multiple first representation data to obtain a candidate representation dataset. Then, based on the candidate representation dataset and the number of second candidate representation data, a second representation dataset is determined. The second representation dataset includes the first representation data excluding the candidate first representation data and neighboring representation data, as well as the number of second candidate representation data. Each data point in the second representation dataset is the second representation data.

[0123] Next, it is determined whether the number of second representation data contained in the second representation dataset is less than or equal to a preset number threshold, that is, whether the number of second representation data is less than or equal to the expected data volume. If it is determined that the number of second representation data contained in the second representation dataset is less than or equal to the preset number threshold, then the second representation dataset is determined as the second sample dataset.

[0124] If it is determined that the number of second representation data contained in the second representation dataset is still greater than the preset number threshold, another candidate first representation data can be determined from the second representation dataset as a candidate second representation data, and the point density of the candidate first representation data can be updated based on the point density of the candidate second representation data. The above operation of determining whether the number of second representation data contained in the second representation dataset is less than or equal to the preset number threshold is repeated until it is determined that the number of second representation data contained in the second representation dataset is less than or equal to the preset number threshold, or the number of repeated operations reaches a predetermined number.

[0125] The following example illustrates the process of determining candidate second characterization data using specific embodiments.

[0126] It is understandable that the second representation dataset is obtained by replacing multiple candidate first representation data and neighboring representation data in the first representation dataset with a number of second candidate representation data. Therefore, the point density of each second representation data in the second representation dataset will change accordingly, and the point density of each second representation data needs to be recalculated in order to determine the second representation data with the highest point density in the second representation dataset, i.e., the candidate second representation data.

[0127] Figure 4 This is a schematic diagram illustrating the process of generating a second representation dataset according to an embodiment of the present invention. (Refer to the following...) Figure 4 An example is provided to illustrate the changes in the point density of each second representation data point in the second representation dataset.

[0128] like Figure 4 As shown in 440, multiple first characterization data include, for example, first characterization data ct1, first characterization data ct2, first characterization data ct3, first characterization data ct4, first characterization data ct5, first characterization data ct6, ...

[0129] For example, using a second distance threshold lc, for each first representation data, based on the second distance threshold lc and the feature vector of the first representation data, the number of neighboring representation data associated with that first representation data is determined from multiple first representation data. Then, the number of neighboring representation data is determined as the point density of that first representation data. Taking first representation data ct1 as an example, for example, based on the second distance threshold lc and the feature vector of the first representation data, the number of neighboring representation data associated with the first representation data ct1 is determined to be 9. These 9 neighboring representation data are located within a density region centered on the first representation data ct1 and with a radius equal to the second distance threshold lc, and include, for example, first representation data ct2, first representation data ct6, etc.

[0130] Next, the point densities corresponding to each of the multiple first representation data are sorted, and based on the point density sorting results, the first representation data with the highest point density among the multiple first representation data is determined as the candidate first representation data. For example, the first representation data with the highest point density among the multiple first representation data is the first representation data ct1, that is, the candidate first representation data is the first representation data ct1.

[0131] Next, based on the point density of the first candidate representation data, the number of second candidate representation data associated with the first candidate representation data is determined. For example, based on the above formula (1), the number of second candidate representation data associated with the first candidate representation data can be determined to be, for example, 4.

[0132] Next, based on the number of second candidate representation data, the candidate first representation data and the neighboring representation data are clustered to obtain a number of second candidate representation data.

[0133] like Figure 4 As shown in 450, for example, based on the number of second candidate representation data, the first representation data ct1 and the neighboring representation data of the first representation data ct1 are clustered to generate four second candidate representation data, such as second candidate representation data cp1, second candidate representation data cp2, second candidate representation data cp3 and second candidate representation data cp4.

[0134] Next, a number of second candidate representation data are used to replace the candidate first representation data and the corresponding neighboring representation data in the multiple first representation data to obtain the second representation dataset.

[0135] like Figure 4 As shown in 460, for example, second candidate representation data cp1 to second candidate representation data cp4 can be used to replace the first representation data ct1 and its neighboring representation data to obtain a second representation dataset. The second representation dataset includes four second candidate representation data and other first representation data besides the first representation data ct1 and its neighboring representation data. Each representation data contained in the second representation dataset is the second representation data.

[0136] During the process of generating the second representation dataset, for first representation data whose distance from the candidate first representation data (i.e., first representation data ct1) is less than or equal to the second distance threshold lc, i.e., the neighboring representation data of the first representation data ct1 (e.g., the first representation data ct2 shown in 440), these neighboring representation data and candidate first representation data will be replaced. In other words, these neighboring representation data and candidate first representation data will not exist in the second representation dataset.

[0137] Furthermore, for first representation data whose distance from candidate first representation data is greater than one second distance threshold lc and less than or equal to two second distance thresholds lc, such as first representation data ct3 shown in 440, since the density region of the candidate first representation data overlaps with the density region of the first representation data ct3, in other words, the candidate first representation data and the first representation data ct3 have the same neighboring representation data (e.g., first representation data ct6). After replacing the candidate first representation data and neighboring representation data in multiple first representation data with several second candidate representation data, the point density of first representation data such as first representation data ct3 in the second representation dataset will be changed by the influence of the candidate first representation data (i.e., first representation data ct1). Therefore, it is necessary to recalculate the point density of these affected first representation data.

[0138] In addition, for first representation data whose distance from the candidate first representation data is greater than twice the second distance threshold lc, such as the first representation data within the density region of the first representation data ct4 shown in 440, since the candidate first representation data and the neighboring representation data are independent of each other, these data will not be affected by the candidate first representation data and the neighboring representation data, and there is no need to calculate the point density of these first representation data again.

[0139] Furthermore, the second representation dataset also includes a number of newly generated second candidate representation data, and the distances between these second candidate representation data and between these second candidate representation data and other first representation data in the second representation dataset besides the candidate first representation data and neighboring representation data will change accordingly (e.g., Figure 4 (As shown in 470). Therefore, it is also necessary to calculate the point density of the newly generated number of second candidate representation data.

[0140] As described above, when determining candidate second representation data, point density calculations can be performed only on the affected first representation data and a certain number of candidate second representation data, instead of calculating the point density of all second representation data in the second representation dataset. This improves data processing efficiency and saves computational resources.

[0141] According to embodiments of the present invention, candidate second characterization data can be determined in the following manner.

[0142] First, identify the affected first characterization data that is associated with the candidate first characterization data from among multiple first characterization data. For example, the first characterization data whose distance from the candidate first characterization data is greater than one second distance threshold and less than or equal to two second distance thresholds can be identified as the affected first characterization data.

[0143] Next, the point density of the first affected representation data and the point density of the number of second candidate representation data are determined respectively.

[0144] It should be noted that, in this embodiment of the invention, when determining the point density of the affected first representation data and the point density of the number of second candidate representation data, for each representation data, based on a second distance threshold and the feature vector of the representation data, the number of neighboring representation data associated with the representation data is determined from the second representation dataset. Then, the number of neighboring representation data is determined as the point density of the representation data.

[0145] Next, based on the point density of the affected first representation data, the point density of the number of second candidate representation data, and the point density of other representation data in the candidate representation dataset besides the affected first representation data, the second representation data with the largest point density in the second representation dataset is determined as the candidate second representation data.

[0146] In some embodiments, the point density of each second representation data in the second representation dataset can be reordered based on an ordered queue to obtain candidate second representation data. See below for reference. Figure 5 An example is provided to illustrate the point density reordering process.

[0147] Figure 5 This is a schematic diagram illustrating the reordering of point densities of each second representation data in the second representation dataset according to an embodiment of the present invention.

[0148] like Figure 5 As shown in 501, for multiple first characterization data, such as first characterization data ct1 to first characterization data ct n n is an integer greater than 1, where n is the first representation data ct1 to the first representation data ct. n After determining the point density, these first representation data can be sorted according to their point density to obtain representation data sequence 51. Then, based on representation data sequence 51, the first representation data with the largest point density among the multiple first representation data can be obtained, i.e., the first representation number ct1, and this first representation number ct1 is used as a candidate first representation data.

[0149] Next, based on the point density of the first candidate representation data, the number of second candidate representation data associated with the first candidate representation data is determined. For example, the number of second candidate representation data associated with the first representation data ct1 can be determined to be m, where m is a positive integer.

[0150] Next, based on the number of second candidate representation data, the candidate first representation data and their corresponding neighboring representation data are clustered to obtain a number of second candidate representation data. For example, by clustering the first representation data ct1 and its corresponding neighboring representation data (e.g., including the first representation data ct1), g First characterization data ct j Clustering is performed on (where g and j are positive integers less than n) to obtain m second candidate representation data, such as second candidate representation data cp1, second candidate representation data cp1, ... ... m .

[0151] Next, the affected first characterization data associated with the candidate first characterization data are identified from among the multiple first characterization data. For example, the affected first characterization data associated with first characterization data ct1 includes first characterization data ct2.

[0152] Next, as Figure 5 As shown in 502, the first characterization data ct1 and the neighboring characterization data corresponding to the first characterization data ct1 (e.g., including the first characterization data ct1) can be removed from the characterization data sequence 51. g First characterization data ct j wait).

[0153] In addition, such as Figure 5 As shown in 503, the affected first representation data, such as first representation data ct2, in the representation data sequence 51 can be moved into the representation data sequence 52 to be sorted. And m second candidate representation data (e.g., second candidate representation data cp1 to second candidate representation data cp2) can be moved into the representation data sequence 52 to be sorted. m The data is moved into the unsorted representation data sequence 52. This facilitates the re-insertion of each representation data in the unsorted representation data sequence 52 back into the representation data sequence 51 after the point density of each representation data in the unsorted representation data sequence 52 has been determined.

[0154] Please continue reading. Figure 5 As shown in 504-506, after determining the point density of each representation data in the representation data sequence 52 to be sorted, each representation data can be inserted back into the representation data sequence 51 in sequence to obtain the reordered representation data sequence.

[0155] Take inserting the first representation data ct2 back into the representation data sequence 51 as an example. For instance, the insertion position of the first representation data ct2 can be determined by comparing the point density of the first representation data ct2 with that of the remaining representation data in the representation data sequence 51.

[0156] If the point density of the first representation data ct2 is greater than or equal to the maximum point density of the remaining representation data in the representation data sequence 51 (corresponding to the point density of the first representation data ct3 in the representation data sequence 51), then the first representation data ct2 is inserted to the left of the first representation data ct3.

[0157] If the point density of the first representation data ct2 is less than or equal to the minimum point density of the remaining representation data in representation data sequence 51 (corresponding to the first representation data ct in representation data sequence 51) n When the point density is (), the first characterization data ct2 is inserted into the first characterization data ct. n On the right side.

[0158] If the point density of the first representation data ct2 is greater than the minimum point density and less than the maximum point density, then the first representation data ct2 can be inserted back into the representation data sequence 51 starting from the position of the point with the maximum point density, for example, by sliding comparison.

[0159] When inserting each representation data back into the representation data sequence 51 using a sliding comparison method, the point density of the representation data to be inserted can satisfy the following conditions.

[0160] N density_i-1 ≥N densitv_current ≥N density_i (2)

[0161] In formula (2), N density_i-1 and N density_i N represents the point density of two adjacent representation data in the representation data sequence 51. density_current Let represent the point density of the representation data to be inserted. Where i = 2, 3, ..., n.

[0162] For example, when the point density of the first representation data ct2 is determined to be between two adjacent representation data (e.g., the first representation data ct4 and the first representation data ct5), k When the point density is between these two adjacent representation data, the first representation data ct2 can be inserted between them, thus completing the insertion of the first representation data ct2 back into the representation data sequence 51.

[0163] It should be noted that, in the process of inserting each representation data back into the representation data sequence 51, in addition to the sliding comparison method, other suitable methods can be used, such as the binary search method. The specific method can be chosen according to actual needs, and no limitation is made here.

[0164] Based on the above method, the point density of each second representation data in the second representation dataset can be reordered, thereby obtaining candidate second representation data according to the reordered ordered queue.

[0165] According to an embodiment of the present invention, in the process of determining candidate first characterization data described above, after obtaining the characterization data sequence, the characterization data sequence can be further divided into multiple first characterization data subsequences for parallel processing according to computing resources, so as to accelerate the efficiency of obtaining candidate first characterization data. This method is particularly suitable for situations where the amount of data in the first characterization dataset is large.

[0166] For example, after sorting multiple first representation data according to their respective point densities to obtain a representation data sequence, the representation data sequence can be split into multiple first representation data subsequences based on computing resources. Each first representation data subsequence includes a predetermined number of first representation data. In one example, each first representation data subsequence can include the same number of first representation data, which ensures that the processing speed of each first representation data subsequence is relatively consistent during parallel computing.

[0167] Next, for each first representation data subsequence, the first representation data with the highest point density among a preset number of first representation data is determined as the first subsequence representation data.

[0168] Next, the first subsequence representation data with the highest point density among multiple first subsequence representation data is selected as candidate first representation data.

[0169] According to an embodiment of the present invention, in the process of reordering the point density of each second representation data in the second representation dataset using an ordered queue to obtain candidate second representation data, the representation data sequence can also be split into multiple first representation data subsequences according to computing resources for parallel reordering, thereby accelerating the efficiency of obtaining candidate second representation data. Each first representation data subsequence includes a preset number of first representation data.

[0170] For example, after identifying the affected first characterization data associated with candidate first characterization data among multiple first characterization data, the affected first characterization data and a number of second candidate characterization data are stored in the sequence of characterization data to be sorted.

[0171] Next, for each first characterization data subsequence, the characterization data related to the candidate first characterization data and neighboring characterization data, as well as the affected first characterization data, are deleted from the first characterization data subsequence to obtain the second characterization data subsequence.

[0172] Next, the point density of the first affected representation data and the point density of the number of second candidate representation data in the sequence of representation data to be sorted are determined. It should be noted that, when determining the point density of the first affected representation data and the point density of the number of second candidate representation data in the sequence of representation data to be sorted, for each representation data, based on a second distance threshold and the feature vector of that representation data, the number of neighboring representation data associated with that representation data is determined from the second representation dataset. Then, the number of neighboring representation data is determined as the point density of that representation data.

[0173] Next, based on the point density of the affected first representation data and the point density of the number of second candidate representation data in the unsorted representation data sequence, as well as the point density of the representation data in the multiple second representation data subsequences, the affected first representation data and the number of second candidate representation data in the unsorted representation data sequence are respectively assigned to multiple second representation data subsequences to obtain multiple third representation data subsequences.

[0174] In this embodiment of the invention, the process of allocating the affected first representation data and a number of second candidate representation data in the representation data sequence to be sorted to multiple second representation data subsequences is similar to the process of inserting each representation data back into the representation data sequence described above, and will not be repeated here.

[0175] Next, for each third representation data subsequence, the representation data with the highest point density in the third representation data subsequence is selected as the second subsequence representation data. Then, the second subsequence representation data with the highest point density among multiple second subsequence representation data is selected as candidate second representation data.

[0176] According to an embodiment of the present invention, in the process of generating a number of second candidate representation data, the object data identifier corresponding to each second candidate representation data can also be determined based on the object data identifier of each object data.

[0177] For example, for each second candidate representation data, candidate first representation data and neighboring representation data associated with the second candidate representation data are determined. Then, based on the object data identifiers corresponding to the multiple object data, the object data identifiers of the candidate first representation data and the object data identifiers of the neighboring representation data are determined.

[0178] Next, based on the object data identifiers of the first candidate representation data and the object data identifiers of neighboring representation data, the object data identifier of the second candidate representation data is determined. Then, the object data identifier of the second candidate representation data is associated with the second candidate representation data.

[0179] For example, the number of object data identifiers belonging to the same data identifier category among the object data identifiers of the candidate first representation data and the object data identifiers of neighboring representation data can be determined. Then, the object data identifier with the largest number is selected as the object data identifier of the second candidate representation data. Here, the process of determining the object data identifier of the second candidate representation data is similar to the process of determining the object data identifier of the first candidate representation data described above, and will not be repeated here.

[0180] Figure 6A This is a diagram illustrating the effect of data classification using relevant technologies. Figure 6B This is a schematic diagram illustrating the effect of data classification based on the technical solution of this invention. See below for reference. Figure 6A and Figure 6B This will illustrate the advantages of the technical solution of the present invention.

[0181] Multiple object data, such as object data P a1 Object data P a2 Object data P a3 ... Wherein, object data P a1 The object data identifier is object data identifier 1, and the object data is P. a2 The object data identifier is object data identifier 2, and the object data is P. a3 The object data identifier is object data identifier 1.

[0182] The object data identifier for each object data item is used to indicate, for example, the category to which the object data belongs. For example, object data P a1 and object data P a3 For example, the category of object data is category 1, and the object data is P. a2 The category is, for example, category 2. It should be noted that, for the sake of simplicity and clarity in illustrating the data classification effect, this embodiment will... Figure 6A and Figure 6B Objects with the same object data identifier are represented by the same symbol.

[0183] With object data P a3 Taking the object data to be classified as an example, relevant techniques such as the k-Nearest Neighbor (KNN) classification algorithm, and the technical solution of this invention combined with the k-Nearest Neighbor classification algorithm are used to classify the object data P. a3 Classify them.

[0184] Figure 6A This is a schematic diagram illustrating the effect of data classification based on the k-nearest neighbor classification algorithm. For example... Figure 6A As shown in 610 and 611, when using the k-nearest neighbor classification algorithm to classify object data P a3When performing classification, using a nearest neighbor value k = 5 (for example only), the nearest neighbor can be determined to be close to the object data P. a3 The five most recent object data entries include three objects with object data identifier 2 and two objects with object data identifier 1. Therefore, object data P can be identified. a3 The classification result is category 2, which is object data P. a3 The object data identifier is object data identifier 2. Clearly, this is the same as object data P. a3 The actual object data identifier (i.e., object data identifier 1) is inconsistent. Therefore, if the object data P is directly classified using the k-nearest neighbor algorithm... a3 When performing classification, the dense object data can cause local noise to affect the classification results, leading to deviations in the classification results.

[0185] Figure 6B This is a schematic diagram illustrating the effect of data classification based on the technical solution of this invention. For example... Figure 6B As shown in 610-630, after processing multiple object data based on the solution of the present invention, a first sample dataset can be obtained. As shown in 630, the first sample dataset includes object data P. a3 Multiple first candidate representation data ct 11 First candidate representation data ct 21 Among them, the first candidate representation data ct 11 It is object data P a1 Generated from the corresponding neighboring object data. First candidate representation data ct 21 It is object data P a2 It is generated from the data of the corresponding neighboring objects.

[0186] Based on the first sample dataset, the k-nearest neighbor classification algorithm is used to classify the object data P. a3 By classifying the data and using the nearest neighbor value k = 5 (for example only), we can determine the relationship between the data and the object P. a3 The five most recent object data entries include four objects with object data identifier 1 and one object data with object data identifier 2. Therefore, object data P can be identified. a3 The classification result is category 1, which is object data P. a3 The object data identifier is set to object data identifier 1. At this time, object data P... a3 The classification results and object data P a3 The actual object data identifier (i.e., object data identifier 1) matches.

[0187] It is understood that in processing multiple object data using the scheme of this invention, a first sample dataset is obtained by using first candidate representation data to replace the corresponding object data and neighboring object data. This simplifies the processing of redundant data in the multiple object data, thereby eliminating the influence of local noise to a certain extent, which helps improve the accuracy of the classification results. Figure 6B The results shown in the figure also illustrate this point.

[0188] Figure 7 This is a flowchart of a training method for a target object detection model according to an embodiment of the present invention.

[0189] like Figure 7 As shown, the training method 700 for the target object detection model includes operations S710 to S720.

[0190] In operation S710, a sample dataset is obtained. In this embodiment of the invention, the sample dataset is obtained using the data processing method described in the above embodiments. The sample dataset includes sample data of the desired amount.

[0191] When operating the S720, the deep learning model is iteratively trained using a sample dataset until the output of the deep learning model meets the iteration stopping condition or the cumulative number of iterations reaches a preset threshold, thus obtaining the target object detection model.

[0192] In this embodiment of the invention, the deep learning model can be trained iteratively multiple times using the aforementioned sample dataset until the output of the deep learning model satisfies the iteration stopping condition or the cumulative number of iterations reaches a preset threshold, thereby obtaining the target object detection model. The iteration stopping condition may, for example, include the difference between the output of the deep learning model and the label information of the sample data satisfying a preset convergence condition.

[0193] In embodiments of the present invention, the sample data includes, for example, at least one of the following: text, image, and video to be detected. A target object detection model is used, for example, to process the sample data to obtain detection results for the sample data.

[0194] In another example, sample data may include text, images, videos, audio, or location information carrying identification data to be classified. This sample data can also be used to train a deep learning model, enabling the trained model to classify the sample data.

[0195] In another example, the sample data may include, for instance, the text to be converted. This sample data can also be used to train a deep learning model, which can then be used to process the text to be converted, producing converted data for that text. The text to be converted may include, for instance, the text to be translated, and the converted data may include, for instance, the translated text. In other words, the trained deep learning model is used to translate the text.

[0196] It should be noted that the type of sample data and the purpose of the trained deep learning model in the embodiments of the present invention are not limited to the above examples, and can be determined according to the actual application scenario, which will not be elaborated here.

[0197] According to embodiments of the present invention, by using a sample dataset with a desired amount of data to train a deep learning model, not only can the training efficiency of the model be improved and computational resources saved, but also the accumulation of local noise caused by excessive sample data can be avoided, thereby improving the accuracy of the model output.

[0198] Figure 8 This is a flowchart of a target object detection method according to an embodiment of the present invention.

[0199] like Figure 8 As shown, the target object detection method 800 includes operation S810.

[0200] In operation S810, the data to be processed is input into the target object detection model to obtain the detection results for the data. The target object detection model is trained using the training method described above.

[0201] According to embodiments of the present invention, the data to be processed includes, for example, at least one of text, images, and videos to be detected. A target object detection model is used, for example, to process the data to obtain detection results for target objects within the data. Of course, the type of data to be processed and the purpose of the target object detection model in these embodiments are not limited to the examples described above, and can be determined according to actual application scenarios, which will not be elaborated further here.

[0202] In the embodiments of the present invention, the target object detection model trained in the above manner is used to detect the target object for the data to be processed, thereby improving the accuracy of target object detection.

[0203] Figure 9 This is a block diagram of a data processing apparatus according to an embodiment of the present invention.

[0204] like Figure 9 As shown, the data processing device 900 includes: a first acquisition module 910, a first determination module 920, a generation module 930, a processing module 940, and a second determination module 950.

[0205] The first acquisition module 910 is used to acquire data from multiple objects.

[0206] The first determining module 920 is used to determine that the number of multiple object data is greater than a preset number threshold. For any candidate object data among the multiple object data, based on the feature vector of the candidate object data and a first distance threshold, it determines the neighboring object data associated with the candidate object data from the multiple object data.

[0207] The generation module 930 is used to generate first candidate representation data in response to the existence of neighboring object data in multiple object data, based on candidate object data and neighboring object data; wherein the first candidate representation data is used to replace candidate object data and neighboring object data.

[0208] The processing module 940 is used to process multiple object data based on candidate object data, neighboring object data and first candidate representation data to obtain the first representation dataset.

[0209] The second determining module 950 is used to determine that the number of first representation data contained in the first representation dataset is less than or equal to a preset number threshold, and to determine the first representation dataset as the first sample dataset.

[0210] Figure 10 This is a block diagram of a training apparatus for a target object detection model according to an embodiment of the present invention.

[0211] like Figure 10 As shown, the training device 1000 for the target object detection model includes a second acquisition module 1010 and a training module 1020.

[0212] The second acquisition module 1010 is used to acquire a sample dataset. The sample dataset is obtained using the data processing device 900 described in the above embodiments.

[0213] The training module 1020 is used to iteratively train the deep learning model using the sample dataset until the output of the deep learning model meets the iteration stopping condition or the cumulative number of iterations reaches a preset threshold, thereby obtaining the target object detection model.

[0214] Figure 11 This is a block diagram of a target object detection device according to an embodiment of the present invention.

[0215] like Figure 11 As shown, the target object detection device 1100 includes: an input module 1110.

[0216] The input module 1110 is used to input the data to be processed into the target object detection model to obtain the target object detection model trained by the training device 1000 of the above embodiment for the data to be processed.

[0217] It should be noted that the implementation methods, technical problems solved, functions achieved, and technical effects of each module / unit / subunit in the device embodiments are the same as or similar to the implementation methods, technical problems solved, functions achieved, and technical effects of each corresponding step in the method embodiments, and will not be repeated here.

[0218] In the technical solution of the present invention, the collection, storage, use, processing, transmission, provision, disclosure and application of data (including but not limited to user personal information) all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0219] In the technical solution of this invention, the authorization or consent of the data owner is obtained before acquiring or collecting relevant data.

[0220] According to embodiments of the present invention, the present invention also provides an electronic device, a readable storage medium, and a computer program product.

[0221] According to an embodiment of the present invention, an electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method described above.

[0222] According to an embodiment of the present invention, a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to perform the method described above.

[0223] According to an embodiment of the present invention, a computer program product includes a computer program that, when executed by a processor, implements the method described above.

[0224] Figure 12 The diagram illustrates a block diagram of an electronic device suitable for implementing a data processing method, a training method for a target object detection model, and a target object detection method according to embodiments of the present invention.

[0225] like Figure 12As shown, an electronic device 1200 according to an embodiment of the present invention includes a processor 1201, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1202 or a program loaded from a storage portion 1208 into a random access memory (RAM) 1203. The processor 1201 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 1201 may also include onboard memory for caching purposes. The processor 1201 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0226] RAM 1203 stores various programs and data required for the operation of electronic device 1200. Processor 1201, ROM 1202, and RAM 1203 are interconnected via bus 1204. Processor 1201 executes various operations of the method flow according to embodiments of the present invention by executing programs in ROM 1202 and / or RAM 1203. It should be noted that the programs may also be stored in one or more memories other than ROM 1202 and RAM 1203. Processor 1201 may also execute various operations of the method flow according to embodiments of the present invention by executing programs stored in said one or more memories.

[0227] According to an embodiment of the present invention, the electronic device 1200 may further include an input / output (I / O) interface 1205, which is also connected to the bus 1204. The electronic device 1200 may also include one or more of the following components connected to the I / O interface 1205: an input section 1206 including a keyboard, mouse, etc.; an output section 1207 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1208 including a hard disk, etc.; and a communication section 1209 including a network interface card such as a LAN card, modem, etc. The communication section 1209 performs communication processing via a network such as the Internet. A drive 1210 is also connected to the I / O interface 1205 as needed. A removable medium 1211, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 1210 as needed so that computer programs read from it can be installed into the storage section 1208 as needed.

[0228] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of the present invention.

[0229] According to embodiments of the present invention, a computer-readable storage medium may be a non-volatile computer-readable storage medium, such as including, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of the present invention, a computer-readable storage medium may include ROM 1202 and / or RAM 1203 and / or one or more memories other than ROM 1202 and RAM 1203 described above.

[0230] Embodiments of the present invention also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code enables the computer system to implement the data processing method, the target object detection model training method, and the target object detection method provided in the embodiments of the present invention.

[0231] When the computer program is executed by the processor 1201, it performs the functions defined in the system / apparatus of this invention. According to embodiments of the invention, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0232] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 1209, and / or installed from the removable medium 1211. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0233] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 1209, and / or installed from the removable medium 1211. When the computer program is executed by the processor 1201, it performs the functions defined in the system of this embodiment of the invention. According to embodiments of the invention, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0234] According to embodiments of the present invention, program code for executing the computer programs provided in the embodiments of the present invention can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0235] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0236] Those skilled in the art will understand that the features described in the various embodiments of the present invention can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, the features described in the various embodiments of the present invention can be combined and / or combined in various ways without departing from the spirit and teachings of the present invention. All such combinations and / or combinations fall within the scope of the present invention.

[0237] The embodiments of the present invention have been described above. However, these embodiments are merely illustrative and not intended to limit the scope of the invention. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of the invention, and all such substitutions and modifications should fall within the scope of the invention.

Claims

1. A data processing method, comprising: Operation S210: Obtain multiple object data, including text type, image type, video type, and audio type; Operation S220: Determine that the number of the plurality of object data is greater than a preset number threshold; For any candidate object data among the plurality of object data, determine the neighboring object data associated with the candidate object data from the plurality of object data based on the feature vector of the candidate object data and a first distance threshold. Operation S230: In response to the existence of neighboring object data among the plurality of object data, first candidate representation data is generated based on the candidate object data and the neighboring object data; wherein, the first candidate representation data is used to replace the candidate object data and the neighboring object data; Operation S240 involves processing the plurality of object data based on the candidate object data, the neighboring object data, and the first candidate representation data to obtain a first representation dataset, including: deleting the candidate object data and the neighboring object data from the plurality of object data to obtain a candidate object dataset; and determining the first representation dataset based on the candidate object dataset and the first candidate representation data; and Operation S250: Determine that the number of first representation data contained in the first representation dataset is less than or equal to the preset number threshold, and determine the first representation dataset as the first sample dataset. If the number of first representation data contained in the first representation dataset is greater than the preset number threshold, another candidate object data is determined from the plurality of object data, and the above candidate object data is updated using the other candidate object data. Operations S220 to S250 are repeated until the number of first representation data contained in the first representation dataset is less than or equal to the preset number threshold.

2. The method according to claim 1, wherein the first representation dataset comprises a plurality of first representation data; In response to detecting that the candidate object dataset does not include object data, and determining that the number of the plurality of first representation data is greater than the preset number threshold, the method further includes: Determining the point density for each of the first representation data includes, for each of the first representation data, determining, based on a second distance threshold and the feature vector of the first representation data, the number of neighboring representation data associated with the first representation data from the plurality of first representation data; and determining the number of neighboring representation data as the point density. The first representation data with the highest point density among the plurality of first representation data is determined as the candidate first representation data; Based on the point density of the first candidate characterization data, determine the number of second candidate characterization data associated with the first candidate characterization data; Based on the number of second candidate representation data, representation data extraction is performed on the plurality of first representation data to obtain a second representation dataset; and If the number of second representation data contained in the second representation dataset is less than or equal to the preset number threshold, the second representation dataset is determined as the second sample dataset.

3. The method according to claim 2, wherein, The step of extracting representation data from the plurality of first representation data based on the number of second candidate representation data to obtain the second representation dataset includes: Based on the feature vector and second distance threshold of the candidate first characterization data, determine the neighboring characterization data associated with the candidate first characterization data from the plurality of first characterization data; Based on the number of second candidate characterization data, clustering is performed on the candidate first characterization data and the neighboring characterization data to obtain the number of second candidate characterization data. Deleting candidate first representation data and neighboring representation data from the plurality of first representation data yields a candidate representation dataset; and The second representation dataset is determined based on the candidate representation dataset and the number of second candidate representation datasets.

4. The method according to claim 3, wherein, In response to determining that the number of second representation data contained in the second representation dataset is greater than the preset number threshold, the method further includes: Identify the affected first characterization data that is associated with the candidate first characterization data among the plurality of first characterization data; Determine the point density of the affected first characterization data and the point density of the number of second candidate characterization data respectively; Based on the point density of the affected first representation data, the point density of the number of second candidate representation data, and the point density of other representation data in the candidate representation dataset besides the affected first representation data, the second representation data with the highest point density in the second representation dataset is determined as the candidate second representation data; and The point density of the candidate first representation data is updated based on the point density of the candidate second representation data, and the operation of determining whether the number of second representation data contained in the second representation dataset is less than or equal to the preset number threshold is repeated.

5. The method according to claim 3, wherein, The step of determining the first representation data with the highest point density among the plurality of first representation data as candidate first representation data includes: Based on the point density corresponding to each of the plurality of first characterization data, the plurality of first characterization data are sorted to obtain a characterization data sequence; Based on computing resources, the representation data sequence is divided into multiple first representation data subsequences; wherein each first representation data subsequence includes a preset number of first representation data. For each first representation data subsequence, the first representation data with the highest point density among the predetermined number of first representation data is determined as the first subsequence representation data; and The first subsequence representation data with the highest point density among multiple first subsequence representation data is selected as candidate first representation data.

6. The method according to claim 5, wherein, In response to determining that the number of second representation data contained in the second representation dataset is greater than the preset number threshold, the method further includes: Identify the affected first characterization data that is associated with the candidate first characterization data among the plurality of first characterization data; The affected first characterization data and the number of second candidate characterization data are stored in a sequence of characterization data to be sorted; For each first characterization data subsequence, delete the characterization data in the first characterization data subsequence that is related to the candidate first characterization data and the neighboring characterization data, as well as the affected first characterization data, to obtain a second characterization data subsequence; Determine the point density of the affected first representation data and the point density of the number of second candidate representation data in the unsorted representation data sequence, respectively; Based on the point density of the affected first characterization data and the point density of the number of second candidate characterization data in the unsorted characterization data sequence, as well as the point density of the characterization data in the multiple second characterization data subsequences, the affected first characterization data and the number of second candidate characterization data in the unsorted characterization data sequence are respectively assigned to multiple second characterization data subsequences to obtain multiple third characterization data subsequences; For each third characterization data subsequence, the characterization data with the highest point density in the third characterization data subsequence is determined as the second subsequence characterization data; The second subsequence representation data with the highest point density among multiple second subsequence representation data is selected as candidate second representation data; and The point density of the candidate first representation data is updated based on the point density of the candidate second representation data, and the operation of determining whether the number of second representation data contained in the second representation dataset is less than or equal to the preset number threshold is repeated.

7. The method according to claim 3, wherein, Each object data includes an object data identifier; the step of extracting representation data from the plurality of first representation data based on the number of second candidate representation data to obtain a second representation dataset further includes: For each second candidate representation data, determine the candidate first representation data and the neighboring representation data associated with the second candidate representation data; Based on the object data identifiers corresponding to each of the multiple object data, determine the object data identifier of the candidate first representation data and the object data identifier of the neighboring representation data; The object data identifier of the second candidate representation data is determined based on the object data identifier of the first candidate representation data and the object data identifier of the neighboring representation data; and Associate the object data identifier of the second candidate representation data with the second candidate representation data.

8. The method according to claim 7, wherein, The step of determining the object data identifier of the second candidate representation data based on the object data identifier of the candidate first representation data and the object data identifier of the neighboring representation data includes: Determine the number of object data identifiers belonging to the same data identifier category among the object data identifiers of the candidate first representation data and the object data identifiers of the neighboring representation data; and The object data identifier with the largest quantity is used as the object data identifier of the second candidate representation data.

9. The method according to any one of claims 2 to 8, wherein, The number of the second candidate representation data is positively correlated with the point density of the first candidate representation data.

10. The method according to any one of claims 1 to 8, further comprising: In response to determining that there is no neighboring object data associated with the candidate object data among the plurality of object data, the candidate object data is determined as the first candidate representation data.

11. The method according to any one of claims 1 to 8, wherein, Each object data includes an object data identifier; the step of generating first candidate representation data based on the candidate object data and the neighboring object data further includes: Based on the object data identifiers corresponding to each of the multiple object data, determine the object data identifiers of the candidate object data and the object data identifiers of the neighboring object data; Based on the object data identifiers of the candidate object data and the object data identifiers of the neighboring object data, the object data identifier of the first candidate representation data is determined; and Associate the object data identifier of the first candidate representation data with the first candidate representation data.

12. The method according to claim 11, wherein, The step of determining the object data identifier of the first candidate representation data based on the object data identifier of the candidate object data and the object data identifier of the neighboring object data includes: Determine the number of object data identifiers belonging to the same data identifier category among the object data identifiers of the candidate object data and the object data identifiers of the neighboring object data; and The object data identifier with the largest quantity is used as the object data identifier of the first candidate representation data.

13. A method for training a target object detection model, comprising: Obtain the sample dataset; The deep learning model is iteratively trained using the sample dataset until the output of the deep learning model meets the iteration stopping condition or the cumulative number of iterations reaches a preset threshold, thereby obtaining the target object detection model. The sample dataset is obtained using the method described in any one of claims 1 to 12.

14. A target object detection method, comprising: Input the data to be processed into the target object detection model to obtain the detection results for the data to be processed; The target object detection model is trained using the method described in claim 13.

15. An electronic device comprising: One or more processors; Memory, used to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors perform the method according to any one of claims 1 to 14.

16. A computer-readable storage medium having stored thereon executable instructions that, when executed by a processor, cause the processor to perform the method according to any one of claims 1 to 14.

Images