Data mining method and device, electronic equipment and storage medium
By encoding and token segmentation of image and video data, combined with similarity analysis methods, the problems of poor data mining effects and high cost in the existing technology are solved, and a more efficient and accurate data mining process is achieved.
Patent Information
- Application Number
- CN202411885984.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-19
- Publication Date
- 2025-04-15
AI Technical Summary
In the scenario where sample data is small and data mining is difficult, the conventional sampling effect is poor, resulting in waste of costs and increased algorithm iteration cycles.
By encoding the image data of the mining target, the first encoding feature is obtained, and the video data of any video frame in the video frame set of the target video is encoded, and the second encoding feature is obtained. Then, the token generator is used to perform token segmentation of the second coded feature, obtain the token feature set, and obtain the data mining results based on the similarity between the first coded feature and the token features in the token feature set.
It improves the accuracy of data mining, reduces labor costs, shortens the iteration cycle of the algorithm, and reduces the cost of sending data standards.
Smart Images
Figure CN119829796A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of computer technology, in particular to the field of artificial intelligence technology, and specifically to a data mining method and device, an electronic device, and a storage medium. Background Art
[0002] With the development of science and technology, the accuracy of algorithm reasoning has become the focus of users. Among them, the quality of data determines the recognition quality of the algorithm. In order to improve the effect of the algorithm, the quality of the data can be continuously improved. For example, the data can be manually sampled and then the sampled data can be sent to the label. However, when the sample data is small and data mining is difficult, the effect of ordinary sampling is unsatisfactory, which will cause cost waste and increase the iteration cycle of the algorithm. Summary of the invention
[0003] The present invention provides a data mining method and device, an electronic device and a storage medium, the main purpose of which is to improve the accuracy of data mining while reducing the iteration cycle of the algorithm and reducing the cost of data submission.
[0004] According to one aspect of the present disclosure, there is provided a data mining method, comprising:
[0005] Encode the image data of the mining target to obtain a first encoding feature, and encode the video data of any video frame in the video frame set of the target video to obtain a second encoding feature;
[0006] Using a Token generator to tokenize the second coding feature, and obtaining a Token feature set corresponding to the second coding feature;
[0007] A data mining result is obtained according to the similarity between the first coding feature and each Token feature in the Token feature set, wherein the data mining result is used to indicate whether any of the video frames includes the mining target.
[0008] According to another aspect of the present disclosure, there is provided a data mining device, comprising:
[0009] A feature acquisition unit, configured to encode the image data of the mining target to obtain a first encoding feature, and to encode the video data of any video frame in the video frame set of the target video to obtain a second encoding feature;
[0010] A feature segmentation unit, configured to perform Token segmentation on the second coding feature by using a Token generator, and obtain a Token feature set corresponding to the second coding feature;
[0011] A data mining unit is used to obtain a data mining result according to the similarity between the first coding feature and each Token feature in the Token feature set, wherein the data mining result is used to indicate whether any of the video frames includes the mining target.
[0012] According to another aspect of the present disclosure, there is provided an electronic device, comprising:
[0013] at least one processor; and
[0014] a memory communicatively connected to the at least one processor; wherein,
[0015] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform any one of the methods described in the above aspect.
[0016] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute any one of the methods described in the above aspect.
[0017] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, wherein when the computer program is executed by a processor, the computer program implements any one of the methods described in the aforementioned aspect.
[0018] In one or more embodiments of the present disclosure, the image data of the mining target is encoded to obtain a first coding feature, and the video data of any video frame in the video frame set of the target video is encoded to obtain a second coding feature; the second coding feature is tokenized using a Token generator to obtain a Token feature set corresponding to the second coding feature; according to the similarity between the first coding feature and each Token feature in the Token feature set, a data mining result is obtained, wherein the data mining result is used to indicate whether any video frame includes the mining target. Therefore, data mining can be performed through the similarity between the first coding feature and each Token feature in the Token feature set, and the matching of the mined video frame with the mining target can be improved, thereby improving the accuracy of data mining, and no human participation is required, thereby reducing labor costs, reducing algorithm iteration cycles, and reducing data submission costs.
[0019] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.
[0021] Figure 1 is a flowchart of a data mining method according to the first embodiment of the present disclosure;
[0022] Figure 2 is a flowchart of a data mining method according to a second embodiment of the present disclosure;
[0023] Figure 3 is a schematic diagram of an example comparison of a video frame and a mining target according to an embodiment of the present disclosure;
[0024] Figure 4 is an exemplary schematic diagram of a data mining method according to an embodiment of the present disclosure;
[0025] Figure 5 is a structural schematic diagram of a data mining device for implementing the data mining method of an embodiment of the present disclosure;
[0026] Figure 6 It is a block diagram of an electronic device used to implement the data mining method of the embodiment of the present disclosure. DETAILED DESCRIPTION
[0027] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.
[0028] The present disclosure is described in detail below with reference to specific embodiments.
[0029] In the first embodiment, if Figure 1 As shown, Figure 1 The present invention is a flowchart of a data mining method according to the first embodiment of the present invention, which can be implemented by a computer program and can be run on a device for data mining. The computer program can be integrated into an application or run as an independent tool application.
[0030] The data mining device may be an electronic device with artificial intelligence interaction capability, including but not limited to: an autonomous vehicle, a wearable device, a handheld device, a personal computer, a tablet computer, a vehicle-mounted device, a smart phone, a computing device or other processing device connected to a wireless modem, etc. In different networks, the terminal may be called by different names, such as: user equipment, access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication device, user agent or user device, cellular phone, cordless phone, personal digital assistant (PDA), fifth generation mobile communication technology (5G) network, fourth generation mobile communication technology (4G) network, third generation mobile communication technology (3G) network or electronic device in future evolution network, etc.
[0031] Specifically, the data mining method includes:
[0032] S101, encoding the image data of the mining target to obtain a first encoding feature, and encoding the video data of any video frame in the video frame set of the target video to obtain a second encoding feature;
[0033] According to some embodiments, the excavation target may be used to indicate an object to be excavated, for example. The name of the excavation target is not limited. For example, the excavation target may also be referred to as a target obstacle, an object to be identified, an obstacle to be identified, etc. The excavation target does not specifically refer to a fixed target. For example, when a modification instruction for the excavation target is received, the excavation target may also change accordingly. For example, when an obstacle corresponding to the excavation target changes, the excavation target may also change accordingly. The excavation target may include, for example, at least one obstacle.
[0034] In some embodiments, the image data of the mining target may be used to indicate data corresponding to the image including the mining target. The image data may be at least one data. The image data of the mining target does not specifically refer to a fixed data. For example, when the mining target changes, the image data of the mining target may also change accordingly. For example, when the method of obtaining the image data of the mining target changes, the image data of the mining target may also change accordingly.
[0035] According to some embodiments, the first coding feature may be used, for example, to indicate a coding feature corresponding to the image data of the mining target. The coding feature may be, for example, a feature obtained after encoding the data. The "first" in the first coding feature is used to distinguish it from the other coding features and does not specifically refer to a fixed coding feature. For example, when the encoder changes, the first coding feature may also change accordingly. For example, when the image data or the mining target changes, the first coding feature may also change accordingly.
[0036] In some embodiments, the target video can be used to indicate the video to be mined, that is, to determine whether each video frame in the video includes the mining target video. The target video does not specifically refer to a fixed video. For example, when the acquisition time point or acquisition time point of the target video changes, the target video can also change accordingly. For example, when the memory size corresponding to the target video changes, the target video can also change accordingly.
[0037] According to some embodiments, the video frame set may be, for example, a collection of at least one video frame of a target video. The video frame set does not specifically refer to a fixed set. For example, when the target video changes, the video frame set may also change accordingly. For example, when the number of video frames corresponding to the video frame set changes, the video frame set may also change accordingly.
[0038] In some embodiments, any video frame may be used to indicate any video frame in a video frame set, for example.
[0039] According to some embodiments, the second coding feature may be used, for example, to indicate a coding feature corresponding to video data of any video frame.
[0040] In some embodiments, image data of the mining target is encoded to obtain a first encoding feature, and video data of any video frame in a set of video frames of the target video is encoded to obtain a second encoding feature.
[0041] S102, using a Token generator to tokenize the second coding feature, and obtaining a Token feature set corresponding to the second coding feature;
[0042] According to some embodiments, the Token generator may be used, for example, to create or generate a Token, wherein the Token generator in the present public embodiment may be used, for example, to perform Token segmentation on a coding feature.
[0043] In some embodiments, the Token feature set refers to a collection of at least one Token feature obtained after segmenting the second coding feature. The Token feature set does not specifically refer to a fixed set. For example, when the number of Token features corresponding to the Token feature set changes, the Token feature set may also change accordingly. For example, when the Token segmentation method corresponding to the Token feature set changes, the Token feature set may also change accordingly.
[0044] In some embodiments, a Token generator may be used to tokenize the second coding feature to obtain a Token feature set corresponding to the second coding feature.
[0045] S103, obtaining a data mining result according to the similarity between the first coding feature and each Token feature in the Token feature set, wherein the data mining result is used to indicate whether any video frame includes a mining target.
[0046] In some embodiments, the similarity is used to indicate the degree of similarity between the first coding feature and each Token feature in the Token feature set. The number of the similarities may be, for example, consistent with the number of Token features in the Token feature set. Different similarities may correspond to different Token features.
[0047] According to some embodiments, the data mining result is used to indicate whether any video frame includes the mining target. For example, the data mining result does not specifically refer to a fixed result. The data mining result may be that any video frame includes the mining target, or the data mining result may be that any video frame does not include the mining target.
[0048] In some embodiments, a data mining result can be obtained based on the similarity between the first coding feature and each Token feature in the Token feature set, wherein the data mining result is used to indicate whether any video frame includes a mining target.
[0049] In one or more embodiments of the present disclosure, the image data of the mining target is encoded to obtain a first coding feature, and the video data of any video frame in the video frame set of the target video is encoded to obtain a second coding feature; the second coding feature is tokenized using a Token generator to obtain a Token feature set corresponding to the second coding feature; according to the similarity between the first coding feature and each Token feature in the Token feature set, a data mining result is obtained, wherein the data mining result is used to indicate whether any video frame includes the mining target. Therefore, data mining can be performed through the similarity between the first coding feature and each Token feature in the Token feature set, and the matching of the mined video frame with the mining target can be improved, thereby improving the accuracy of data mining, and no human participation is required, reducing labor costs, reducing algorithm iteration cycles, and reducing data submission costs. In addition, there is no need to perform biased learning on the data mining scenario, which can reduce the overfitting caused by learning the mining scenario and reduce the impact on the reasoning ability when reasoning on other reasoning scenarios.
[0050] See also Figure 2 , Figure 2 is a flow chart of a data mining method according to the second embodiment of the present disclosure. Specifically,
[0051] S201, encoding the image data of the mining target to obtain a first encoding feature, and encoding the video data of any video frame in the video frame set of the target video to obtain a second encoding feature;
[0052] The relevant process has been described above and will not be repeated here.
[0053] In some embodiments, the technical solution of the present disclosure can be applied to data mining scenarios such as highways and traffic control, and specifically, it can be an end-to-end target scene data mining scenario.
[0054] According to some embodiments, the excavation target may be used, for example, to indicate a minimum area that includes a target obstacle.
[0055] In some embodiments, the target video may be, for example, a video to be mined, and the target video may be, for example, a video for which it is necessary to determine whether it includes a mining target. The method for obtaining the target video is not limited. For example, the target video may be sent from another electronic device to the execution subject of the embodiment of the present disclosure, or may be a video downloaded from a web page.
[0056] In some embodiments, the target video may be, for example, an original video captured at a work site and sent to an execution subject of the disclosed embodiment. The size of each video frame in the target video may be, for example, larger than the size of the mining target. The size comparison between the target video and the mining target may be, for example, Figure 3 shown.
[0057] in, Figure 4 is an exemplary schematic diagram of a data mining method according to an embodiment of the present disclosure, Figure 4 As shown, before encoding the image data or video data, for example, feature extraction can be performed on the image data and the video data. For example, a feature extractor can be used to extract features. The feature extractor can be, for example, a feature extraction module trained with a large amount of unlabeled data.
[0058] The image data of the mining target can be encoded to obtain the first encoding feature, and the video data of any video frame in the video frame set of the target video can be encoded to obtain the second encoding feature. The execution order of obtaining the first encoding feature and obtaining the second encoding feature is not limited, and the same encoder can be used to obtain.
[0059] The method of encoding the image data of the mining target to obtain a first encoding feature and encoding the video data of any video frame in the video frame set of the target video to obtain a second encoding feature includes:
[0060] Encoding the image data of the mining target using an encoder corresponding to the mining target to obtain a first encoding feature;
[0061] Analyze the target video to obtain a set of video frames corresponding to the target video;
[0062] The encoder is used to encode the video data of any video frame in the video frame set to obtain the second encoding feature. Therefore, the same encoder can be used to encode the image data and the video data, reducing the situation where different encoders encode the image data and the video data, resulting in inaccurate encoding of the encoding feature, improving the consistency of the first encoding feature and the second encoding feature, improving the accuracy of similarity acquisition, and improving the accuracy of data mining.
[0063] In some embodiments, the encoder of the embodiment of the present disclosure may be, for example, an encoder corresponding to the mining target. The encoder may be, for example, an encoder corresponding to the mining target, and specifically, different mining targets may correspond to different encoders.
[0064] In some embodiments, the target video is parsed to obtain a set of video frames corresponding to the target video. For example, the target video may be parsed into offline frames to obtain a set of video frames corresponding to the target video.
[0065] S202, obtaining a segmentation method corresponding to the target video, wherein the segmentation method includes performing Token segmentation on the second coding feature according to a preset feature size or performing Token segmentation on the second coding feature according to a preset feature number;
[0066] The relevant process has been described above and will not be repeated here.
[0067] According to some embodiments, the segmentation method may be used to indicate the method of segmenting the second feature code, for example, and the segmentation method may include segmenting the second code feature into tokens according to a preset feature size or segmenting the second code feature into tokens according to a preset feature number.
[0068] According to some embodiments, a segmentation method corresponding to the target video may be obtained.
[0069] In some embodiments, the segmentation method may be determined, for example, according to a method determination instruction, or according to video information of the target video, which is not limited in the embodiments of the present disclosure.
[0070] S203, tokenize the second coding feature using a segmentation method and a Token generator to obtain a Token feature set corresponding to the second coding feature;
[0071] The relevant process has been described above and will not be repeated here.
[0072] In some embodiments, for example, a Token generator may be used to tokenize the second coding feature, and divide the second coding feature into N*M small Token features.
[0073] According to some embodiments, the second coding feature may be, for example, a coding feature corresponding to any video frame. When the segmentation method is to tokenize the second coding feature according to a preset feature size, the second coding feature may be segmented into a plurality of N*M small Token features.
[0074] S204, obtaining a data mining result according to the similarity between the first coding feature and each Token feature in the Token feature set, wherein the data mining result is used to indicate whether any video frame includes a mining target.
[0075] The relevant process has been described above and will not be repeated here.
[0076] Among them, when the first coding feature and the Token feature set are obtained, the similarity between the first coding feature and each Token feature in the Token feature set can be obtained, that is, at least one similarity can be obtained, wherein the number of similarities of at least one similarity can be consistent with the number of Token features in the Token feature set.
[0077] According to some embodiments, when at least one similarity is obtained, a data mining result may be obtained based on the at least one similarity.
[0078] In some embodiments, for example, a similarity comparator may be used to perform a similarity comparison between the first coding feature and each Token feature in the Token feature set to obtain a data mining result, that is, an output result.
[0079] According to some embodiments, obtaining a data mining result according to the similarity between the first coding feature and each Token feature in the Token feature set includes:
[0080] Obtain the first coding feature and compare the similarity of each Token feature in the Token feature set to obtain at least one similarity;
[0081] Accumulate at least one similarity to obtain a similarity corresponding to any video frame;
[0082] When the similarity is greater than a first similarity threshold, obtaining a data mining result as a first data mining result, where the first data mining result is used to indicate that any video frame includes a mining target;
[0083] When the similarity is less than the second similarity threshold, the data mining result is obtained as the second data mining result, and the second data mining result is used to indicate that any video frame does not include the mining target, wherein the second similarity threshold is less than or equal to the first similarity threshold. Therefore, the similarity corresponding to each Token feature can be accumulated to determine whether the video frame includes the mining target, which can improve the accuracy of mining the mining target without human participation and reduce the data mining cost.
[0084] In some embodiments, each Token feature in the N*M blocks of small Token features can be compared with the first coding feature for similarity, and the similarities obtained from each Token feature are accumulated to obtain a total similarity. When the total similarity is greater than the first similarity threshold, it is determined that the video frame has an excavation target. When the total similarity is less than the second similarity threshold, it is determined that the video frame does not have an excavation target. The first similarity threshold can be the same as the second similarity threshold, and the first similarity threshold can also be greater than the second similarity threshold. For example, the first similarity threshold and the second similarity threshold can both be 0.4, or the first similarity threshold can be 0.6 and the second similarity threshold can be 0.5. The embodiments disclosed herein are not limited to this.
[0085] According to some embodiments, further comprising:
[0086] Obtain current data mining demand information;
[0087] According to the current data mining requirement information, the first similarity threshold and the second similarity threshold are obtained. Therefore, the similarity threshold corresponding to the current data mining requirement information can be determined, which can improve the matching of the similarity threshold with the current data mining requirement information and improve the convenience of data mining.
[0088] According to some embodiments, the current data mining requirement information may be, for example, identifying a video frame including a sanitation worker in the target video. The current data mining requirement information may also be, for example, identifying a video frame including a lost mobile phone in the target video.
[0089] According to some embodiments, obtaining a data mining result according to the similarity between the first coding feature and each Token feature in the Token feature set includes:
[0090] Obtain the first coding feature and compare the similarity of each Token feature in the Token feature set to obtain at least one similarity;
[0091] Obtaining the number of similarities whose similarity is greater than a third similarity threshold value among at least one similarity;
[0092] When the number of similarities is greater than a first number threshold, obtaining a data mining result as a first data mining result, the first data mining result is used to indicate that any video frame includes a mining target;
[0093] When the number of similarities is greater than the second number threshold, the data mining result is obtained as the second data mining result, and the second data mining result is used to indicate that any video frame does not include the mining target, wherein the second number threshold is less than or equal to the first number threshold. Therefore, the data mining result can be determined according to the number of similarities, which can improve the accuracy of obtaining the data mining result.
[0094] In some embodiments, the third similarity threshold does not specifically refer to a fixed threshold. For example, when a modification instruction for the third similarity threshold is received, the third similarity threshold may also be changed accordingly.
[0095] According to some embodiments, further comprising:
[0096] Obtain current data mining demand information;
[0097] According to current data mining requirement information, a first similarity threshold and a second similarity threshold are obtained.
[0098] According to some embodiments, further comprising:
[0099] A subset of video frames including the mining target is obtained from the video frame set, and the subset of video frames is sent for labeling. Therefore, the subset of video frames including the mining target can be sent for labeling, which reduces the human participation process, reduces labor costs, and reduces the iteration cycle of the algorithm. In addition, in data mining application scenarios with few samples or difficult data collection, there is no need to bias learning for such scenarios, which reduces overfitting for such scenarios and the decline in recognition ability for other scenarios.
[0100] In one or related embodiments, a segmentation method corresponding to the target video can be obtained, wherein the segmentation method includes Token segmentation of the second coding feature according to a preset feature size or Token segmentation of the second coding feature according to a preset feature quantity, and the second coding feature is tokenized using the segmentation method and a Token generator to obtain a Token feature set corresponding to the second coding feature, which can improve the matching of the segmentation method with the second coding feature, improve the matching of the Token feature set with the current mining scenario, and improve the accuracy of data mining.
[0101] The following are embodiments of the device disclosed herein, which can be used to execute the method embodiments disclosed herein. For details not disclosed in the device embodiments disclosed herein, please refer to the method embodiments disclosed herein.
[0102] See also Figure 5 , which shows a schematic diagram of the structure of a data mining device for implementing the data mining method of the embodiment of the present disclosure. The data mining device can be implemented as all or part of the device through software, hardware or a combination of both. The data mining device 500 includes a feature acquisition unit 501, a feature segmentation unit 502 and a data mining unit 503, wherein:
[0103] The feature acquisition unit 501 is used to encode the image data of the mining target to obtain a first encoding feature, and encode the video data of any video frame in the video frame set of the target video to obtain a second encoding feature;
[0104] A feature segmentation unit 502 is used to perform Token segmentation on the second coding feature using a Token generator to obtain a Token feature set corresponding to the second coding feature;
[0105] The data mining unit 503 is used to obtain a data mining result according to the similarity between the first coding feature and each Token feature in the Token feature set, wherein the data mining result is used to indicate whether any video frame includes a mining target.
[0106] According to some embodiments, the feature acquisition unit 501 is used to encode the image data of the mining target, obtain the first encoding feature, and encode the video data of any video frame in the video frame set of the target video, and when obtaining the second encoding feature, specifically for:
[0107] Encoding the image data of the mining target using an encoder corresponding to the mining target to obtain a first encoding feature;
[0108] Analyze the target video to obtain a set of video frames corresponding to the target video;
[0109] An encoder is used to encode video data of any video frame in the video frame set to obtain a second encoding feature.
[0110] According to some embodiments, the feature segmentation unit 502 is used to perform Token segmentation on the second coding feature and obtain a Token feature set corresponding to the second coding feature, specifically for:
[0111] Obtaining a segmentation method corresponding to the target video, wherein the segmentation method includes performing Token segmentation on the second coding feature according to a preset feature size or performing Token segmentation on the second coding feature according to a preset feature quantity;
[0112] The second coding feature is tokenized using a segmentation method and a Token generator to obtain a Token feature set corresponding to the second coding feature.
[0113] According to some embodiments, the data mining unit 503 is used to obtain the data mining result according to the similarity between the first coding feature and each token feature in the token feature set, and is specifically used to:
[0114] Obtain the first coding feature and compare the similarity of each Token feature in the Token feature set to obtain at least one similarity;
[0115] Accumulate at least one similarity to obtain a similarity corresponding to any video frame;
[0116] When the similarity is greater than a first similarity threshold, obtaining a data mining result as a first data mining result, where the first data mining result is used to indicate that any video frame includes a mining target;
[0117] When the similarity is less than a second similarity threshold, the data mining result is obtained as a second data mining result, and the second data mining result is used to indicate that any video frame does not include a mining target, wherein the second similarity threshold is less than or equal to the first similarity threshold.
[0118] According to some embodiments, the data mining unit 503 is further configured to:
[0119] Obtain current data mining demand information;
[0120] According to current data mining requirement information, a first similarity threshold and a second similarity threshold are obtained.
[0121] According to some embodiments, the data mining unit 503 is used to obtain the data mining result according to the similarity between the first coding feature and each token feature in the token feature set, and is specifically used to:
[0122] Obtain the first coding feature and compare the similarity of each Token feature in the Token feature set to obtain at least one similarity;
[0123] Obtaining the number of similarities whose similarity is greater than a third similarity threshold value among at least one similarity;
[0124] When the number of similarities is greater than a first number threshold, obtaining a data mining result as a first data mining result, the first data mining result is used to indicate that any video frame includes a mining target;
[0125] When the similarity quantity is greater than a second quantity threshold, the data mining result is obtained as a second data mining result, and the second data mining result is used to indicate that any video frame does not include a mining target, wherein the second quantity threshold is less than or equal to the first quantity threshold.
[0126] According to some embodiments, the data mining unit 503 is further configured to:
[0127] A subset of video frames including the mining target is obtained from the video frame set, and the subset of video frames is subjected to labeling processing.
[0128] It should be noted that the data mining device provided in the above embodiment only uses the division of the above functional modules as an example when executing the data mining method. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the data mining device provided in the above embodiment and the data mining method embodiment belong to the same concept, and the implementation process thereof is detailed in the method embodiment, which will not be repeated here.
[0129] The serial numbers of the above-mentioned embodiments of the present disclosure are only for description and do not represent the advantages or disadvantages of the embodiments.
[0130] In summary, the device provided by the embodiment of the present disclosure includes a feature acquisition unit, which is used to encode the image data of the mining target to obtain the first coding feature, and encode the video data of any video frame in the video frame set of the target video to obtain the second coding feature; a feature segmentation unit, which is used to tokenize the second coding feature using a Token generator to obtain a Token feature set corresponding to the second coding feature; and a data mining unit, which is used to obtain data mining results based on the similarity between the first coding feature and each Token feature in the Token feature set, wherein the data mining result is used to indicate whether any video frame includes the mining target. Therefore, data mining can be performed through the similarity between the first coding feature and each Token feature in the Token feature set, and the matching of the mined video frame with the mining target can be improved, thereby improving the accuracy of data mining, and no human participation is required, thereby reducing labor costs, reducing the iteration cycle of the algorithm, and reducing the cost of data submission.
[0131] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.
[0132] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.
[0133] Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement an embodiment of the present disclosure is shown. The components, their connections and relationships, and their functions shown herein are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.
[0134] like Figure 6 As shown, the electronic device includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the electronic device can also be stored. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.
[0135] Multiple components in the electronic device are connected to the I / O interface 605, including: an input unit 606, such as a keyboard, a mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a disk, an optical disk, etc.; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows the electronic device to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0136] The computing unit 601 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 601 performs the various methods and processes described above, such as data mining methods. For example, in some embodiments, the data mining method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on an electronic device via ROM 602 and / or a communication unit 609. When the computer program is loaded into RAM 603 and executed by the computing unit 601, one or more steps of the data mining method described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform the data mining method in any other appropriate manner (e.g., by means of firmware).
[0137] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0138] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or electronic device.
[0139] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0140] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).
[0141] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data electronic device), or a computing system that includes middleware components (e.g., an application electronic device), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.
[0142] A computer system may include a client and an electronic device. The client and the electronic device are generally remote from each other and usually interact through a communication network. The relationship between the client and the electronic device is generated by computer programs running on the corresponding computers and having a client-electronic device relationship with each other. The electronic device may be a cloud electronic device, also known as a cloud computing electronic device or a cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The electronic device may also be an electronic device of a distributed system, or an electronic device combined with a blockchain.
[0143] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.
[0144] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.
Claims
1. A data mining method, comprising: Encode the image data of the mining target to obtain a first encoding feature, and encode the video data of any video frame in the video frame set of the target video to obtain a second encoding feature; Using a Token generator to tokenize the second coding feature, and obtaining a Token feature set corresponding to the second coding feature; A data mining result is obtained according to the similarity between the first coding feature and each Token feature in the Token feature set, wherein the data mining result is used to indicate whether any of the video frames includes the mining target.
2. The method according to claim 1, wherein: The step of encoding the image data of the mining target to obtain a first encoding feature, and encoding the video data of any video frame in the video frame set of the target video to obtain a second encoding feature includes: Encoding the image data of the mining target using an encoder corresponding to the mining target to obtain a first encoding feature; Analyze the target video to obtain a set of video frames corresponding to the target video; The encoder is used to encode video data of any video frame in the video frame set to obtain a second encoding feature.
3. The method according to claim 1, wherein: The performing Token segmentation on the second coding feature to obtain a Token feature set corresponding to the second coding feature includes: Obtaining a segmentation method corresponding to the target video, wherein the segmentation method includes performing Token segmentation on the second coding feature according to a preset feature size or performing Token segmentation on the second coding feature according to a preset feature quantity; The second coding feature is tokenized using the segmentation method and the Token generator to obtain a Token feature set corresponding to the second coding feature.
4. The method according to claim 1, wherein: The obtaining of data mining results according to the similarity between the first coding feature and each Token feature in the Token feature set includes: Obtaining the first coding feature and performing similarity comparison with each Token feature in the Token feature set to obtain at least one similarity; Accumulating the at least one similarity to obtain a similarity corresponding to any one of the video frames; In the case where the similarity is greater than a first similarity threshold, obtaining a data mining result as a first data mining result, wherein the first data mining result is used to indicate that any one of the video frames includes the mining target; When the similarity is less than a second similarity threshold, the data mining result is obtained as a second data mining result, and the second data mining result is used to indicate that any video frame does not include the mining target, wherein the second similarity threshold is less than or equal to the first similarity threshold.
5. The method according to claim 4, further comprising: Obtain current data mining demand information; The first similarity threshold and the second similarity threshold are acquired according to the current data mining requirement information.
6. The method according to claim 1, wherein: The obtaining of data mining results according to the similarity between the first coding feature and each Token feature in the Token feature set includes: Obtaining the first coding feature and performing similarity comparison with each Token feature in the Token feature set to obtain at least one similarity; Obtaining the number of similarities among the at least one similarity whose similarity is greater than a third similarity threshold; When the number of similarities is greater than a first number threshold, obtaining a data mining result as a first data mining result, wherein the first data mining result is used to indicate that any one of the video frames includes the mining target; When the similarity quantity is greater than a second quantity threshold, the data mining result obtained is a second data mining result, and the second data mining result is used to indicate that any video frame does not include the mining target, wherein the second quantity threshold is less than or equal to the first quantity threshold.
7. The method according to claim 4 or 6, further comprising: A video frame subset including the mining target in the video frame set is obtained, and the video frame subset is subjected to labeling processing.
8. A data mining device, comprising: A feature acquisition unit, configured to encode the image data of the mining target to obtain a first encoding feature, and to encode the video data of any video frame in the video frame set of the target video to obtain a second encoding feature; A feature segmentation unit, configured to perform Token segmentation on the second coding feature using a Token generator, and obtain a Token feature set corresponding to the second coding feature; A data mining unit is used to obtain a data mining result according to the similarity between the first coding feature and each Token feature in the Token feature set, wherein the data mining result is used to indicate whether any of the video frames includes the mining target.
9. The device according to claim 8, wherein: The feature acquisition unit is used to encode the image data of the mining target, obtain the first encoding feature, and encode the video data of any video frame in the video frame set of the target video to obtain the second encoding feature, specifically for: Encoding the image data of the mining target using an encoder corresponding to the mining target to obtain a first encoding feature; Analyze the target video to obtain a set of video frames corresponding to the target video; The encoder is used to encode video data of any video frame in the video frame set to obtain a second encoding feature.
10. The device according to claim 8, wherein: The feature segmentation unit is used to perform Token segmentation on the second coding feature and obtain a Token feature set corresponding to the second coding feature, specifically for: Obtaining a segmentation method corresponding to the target video, wherein the segmentation method includes performing Token segmentation on the second coding feature according to a preset feature size or performing Token segmentation on the second coding feature according to a preset feature quantity; The second coding feature is tokenized using the segmentation method and the Token generator to obtain a Token feature set corresponding to the second coding feature.
11. The device according to claim 8, wherein: The data mining unit is used to obtain the data mining result according to the similarity between the first coding feature and each token feature in the token feature set, specifically to: Obtaining the first coding feature and performing similarity comparison with each Token feature in the Token feature set to obtain at least one similarity; Accumulating the at least one similarity to obtain a similarity corresponding to any one of the video frames; In the case where the similarity is greater than a first similarity threshold, obtaining a data mining result as a first data mining result, wherein the first data mining result is used to indicate that any one of the video frames includes the mining target; When the similarity is less than a second similarity threshold, the data mining result is obtained as a second data mining result, and the second data mining result is used to indicate that any video frame does not include the mining target, wherein the second similarity threshold is less than or equal to the first similarity threshold.
12. The device according to claim 11, wherein the data mining unit is further configured to: Obtain current data mining demand information; The first similarity threshold and the second similarity threshold are acquired according to the current data mining requirement information.
13. The device according to claim 8, wherein: The data mining unit is used to obtain the data mining result according to the similarity between the first coding feature and each token feature in the token feature set, specifically to: Obtaining the first coding feature and performing similarity comparison with each Token feature in the Token feature set to obtain at least one similarity; Acquire the number of similarities in the at least one similarity whose similarity is greater than a third similarity threshold; when the number of similarities is greater than a first number threshold, acquire the data mining result as a first data mining result, wherein the first data mining result is used to indicate that any one of the video frames includes the mining target; When the similarity quantity is greater than a second quantity threshold, the data mining result obtained is a second data mining result, and the second data mining result is used to indicate that any video frame does not include the mining target, wherein the second quantity threshold is less than or equal to the first quantity threshold.
14. The device according to claim 11 or 13, wherein the data mining unit is further used for: A video frame subset including the mining target in the video frame set is obtained, and the video frame subset is subjected to labeling processing.
15. An electronic device, comprising: at least one processor; as well as A memory communicatively connected to the at least one processor; characterized in that The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.
16. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-7.
17. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 7.