Information detection method, device, electronic device and storage medium

Abstract

The present disclosure relates to an information detection method, device, electronic device and storage medium, and relates to the field of data processing technology. The embodiments of the present disclosure at least solve the problem in the related art that the inaccurate classification model leads to inaccurate results in detecting whether different information representation methods of the same object are consistent. The method includes: obtaining first feature data and second feature data of the target object; searching for a target preset object corresponding to the target object in a preset object set according to the first feature data; obtaining a second modal similarity according to the fourth feature data and the second feature data of the target preset object; if the second modal similarity is greater than the second threshold, determining that the first modal data corresponding to the target object matches the second modal data corresponding to the target object. Comparing the single modal data of the target object separately to determine whether the different modal data of the same target object match can improve the accuracy of detecting whether the representation methods are consistent.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, and in particular to an information detection method, device, electronic device and storage medium. Background Art

[0002] In the information publishing scenario, the same object can be represented by at least two of the following information representation methods: text, video, image, and voice. In the fields of search, recommendation, and advertising, the information representation method that covers less information is used to determine whether the corresponding same information meets user needs. Therefore, the consistency of at least two information representation methods corresponding to the same information is very important.

[0003] In the related art, assuming that the same object is represented by two information methods, target text and target video, in order to detect whether the target text is consistent with the target video, a data set is first obtained, which includes positive samples in which the text is consistent with the video, and negative samples in which the text is inconsistent with the video. The data set is then input into an encoder to obtain feature data, and then a classification model is trained based on the feature data input and binary cross entropy. Finally, based on the classification model, whether the target text is consistent with the target video is detected.

[0004] However, the above classification model is highly dependent on the data set, and the data set required for the training method of the above classification model is high-quality confidence samples obtained through manual collection. It is difficult to collect high-quality confidence samples, which leads to inaccurate classification models, and further leads to inaccurate results in detecting whether different information representation methods of the same object are consistent. Summary of the invention

[0005] The present disclosure provides an information detection method, device, electronic device and storage medium to at least solve the problem in the related art that the difficulty in collecting high-quality confidence samples leads to inaccurate classification models, which in turn leads to inaccurate results in detecting whether different information representation methods of the same object are consistent. The technical solution of the present disclosure is as follows:

[0006] According to a first aspect of an embodiment of the present disclosure, a method for detecting information consistency is provided, comprising: obtaining first feature data of a target object according to a first single-modal dual-tower model, the first feature data being feature data of first modal data corresponding to the target object, and the target object is a multimedia resource object; obtaining second feature data of the target object according to a second single-modal dual-tower model, the second feature data being feature data of second modal data corresponding to the target object; searching for a target preset object corresponding to the target object in a preset object set according to the first feature data, the first modal similarity between third feature data of the target preset object and the first feature data being greater than or equal to a first threshold, the third feature data being feature data of the first modal data corresponding to the target preset object; the preset object set comprising at least one preset object, the feature data of the first modal data corresponding to the preset object matching the feature data of the second modal data corresponding to the preset object; obtaining second modal similarity between the target preset object and the target object according to fourth feature data and second feature data of the target preset object, the fourth feature data being feature data of the second modal data corresponding to the target preset object; and determining that the first modal data corresponding to the target object matches the second modal data corresponding to the target object when the second modal similarity is greater than the second threshold.

[0007] Optionally, when the number of target preset objects is greater than 1, the above-mentioned “obtaining the second modal similarity between the target preset object and the target object based on the fourth feature data and the second feature data of the target preset object” includes: obtaining the third modal similarity between each target preset object and the target object based on the fourth feature data and the second feature data of each target preset object; and determining the average value of the third modal similarity between each target preset object and the target object as the second modal similarity between the target preset object and the target object.

[0008] Optionally, the above-mentioned information detection method also includes: according to the first unimodal twin-tower model, before obtaining the first feature data of the target object, determining the first unimodal twin-tower model according to the data type of the first modal data corresponding to the target object, the first unimodal twin-tower model is trained based on the first sample data, and the data type of the first sample data is the same as the data type of the first modal data; according to the second unimodal twin-tower model, before obtaining the second feature data of the target object, determining the second unimodal twin-tower model according to the data type of the second modal data corresponding to the target object, the second unimodal twin-tower model is trained based on the second sample data, and the data type of the second sample data is the same as the data type of the second modal data.

[0009] Optionally, the above-mentioned information detection method also includes: according to the first feature data, before searching for the target preset object corresponding to the target object in the preset object set, according to the first single-modal dual-tower model, extracting and storing the feature data of the first modal data corresponding to each preset object in the preset object set; according to the second single-modal dual-tower model, extracting and storing the feature data corresponding to the preset object set and the second modal data corresponding to each preset object.

[0010] Optionally, the above-mentioned information detection method also includes: when the second modality similarity is greater than a second threshold, after determining that the first modality data corresponding to the target object matches the second modality data corresponding to the target object, adding the target object to the preset object set and recording the first feature data and the second feature data.

[0011] Optionally, in the above information detection method, the first modal data is any one of the following: text data, video data, audio data and image data; the second modal data is any one of the following: text data, video data, audio data and image data.

[0012] According to a second aspect of an embodiment of the present disclosure, an information detection device is provided, including: an acquisition unit, a search unit and a determination unit; the acquisition unit is used to acquire first feature data of a target object according to a first unimodal dual-tower model, the first feature data being feature data of first modal data corresponding to the target object, and the target object is a multimedia resource object; the acquisition unit is also used to acquire second feature data of the target object according to a second unimodal dual-tower model, the second feature data being feature data of second modal data corresponding to the target object; the search unit is used to search for a target preset object corresponding to the target object in a preset object set according to the first feature data acquired by the acquisition unit, and a first modal similarity between the third feature data of the target preset object and the first feature data is greater than or equal to a first threshold, and the third feature data is feature data of the first modal data corresponding to the target preset object; the preset object set includes at least one preset object, and the feature data of the first modal data corresponding to the preset object matches the feature data of the second modal data corresponding to the preset object; the acquisition unit is further used to obtain the second modal similarity between the target preset object and the target object according to the fourth feature data and the second feature data of the target preset object found by the search unit, and the fourth feature data is the feature data of the second modal data corresponding to the target preset object; the determination unit is used to determine that the first modal data corresponding to the target object matches the second modal data corresponding to the target object when the second modal similarity obtained by the acquisition unit is greater than the second threshold.

[0013] Optionally, when the number of target preset objects is greater than 1, the above-mentioned acquisition unit is specifically used to: obtain the third modal similarity between each target preset object and the target object based on the fourth feature data and the second feature data of each target preset object; and determine the average value of the third modal similarity between each target preset object and the target object as the second modal similarity between the target preset object and the target object.

[0014] Optionally, the above-mentioned information detection device also includes: a determination unit, which is also used for the acquisition unit to determine the first unimodal twin-tower model according to the data type of the first modal data corresponding to the target object before acquiring the first feature data of the target object according to the first unimodal twin-tower model, and the first unimodal twin-tower model is trained based on the first sample data, and the data type of the first sample data is the same as the data type of the first modal data; determine the second unimodal twin-tower model according to the data type of the second modal data corresponding to the target object, and the second unimodal twin-tower model is trained based on the second sample data, and the data type of the second sample data is the same as the data type of the second modal data.

[0015] Optionally, the above-mentioned information detection device also includes an extraction unit; the processing unit is also used for the search unit to extract and store the feature data of the first modal data corresponding to each preset object in the preset object set according to the first single-modal dual-tower model before searching for the target preset object corresponding to the target object in the preset object set according to the first feature data; and to extract and store the feature data corresponding to the preset object set and the second modal data corresponding to each preset object according to the second single-modal dual-tower model.

[0016] Optionally, the above-mentioned information detection device also includes a adding unit; the adding unit is used to add the target object to the preset object set and record the first feature data and the second feature data after the determination unit determines that the first modal data corresponding to the target object matches the second modal data corresponding to the target object when the second modal similarity is greater than a second threshold.

[0017] Optionally, in the above-mentioned information detection device, the first modal data is any one of the following: text data, video data, audio data and image data; the second modal data is any one of the following: text data, video data, audio data and image data.

[0018] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a processor; a memory for storing processor executable instructions; wherein the processor is configured to execute the above instructions to implement an information detection method provided in the first aspect and any possible design thereof.

[0019] According to a fourth aspect of an embodiment of the present disclosure, a readable storage medium is provided. When instructions in the readable storage medium are executed by a processor, the information detection method provided in the first aspect and any possible design thereof can be implemented.

[0020] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, including a computer program / instruction, which, when executed by a processor, implements the information detection method provided in the first aspect and any possible design thereof.

[0021] The technical solution provided by the present disclosure brings at least the following beneficial effects: first, according to the first single-modal dual-tower model, the first feature data of the target object is obtained, the first feature data is the feature data of the first modal data corresponding to the target object, and the target object is a multimedia resource object; according to the second single-modal dual-tower model, the second feature data of the target object is obtained, the second feature data is the feature data of the second modal data corresponding to the target object; then, according to the first feature data, the target preset object corresponding to the target object is searched in the preset object set, the first modal similarity between the third feature data of the target preset object and the first feature data is greater than or equal to the first threshold, the third feature data is the feature data of the first modal data corresponding to the target preset object; then, according to the fourth feature data and the second feature data of the target preset object, the second modal similarity between the target preset object and the target object is obtained, the fourth feature data is the feature data of the second modal data corresponding to the target preset object; finally, when the second modal similarity is greater than the second threshold, it is determined that the first modal data corresponding to the target object matches the second modal data corresponding to the target object. In this way, for different modal data corresponding to the same target object, the first feature data and the second feature data are extracted respectively, and then the target preset object corresponding to the third feature data similar to the first feature data is found under the same modality, and then the fourth feature data of the second modal data corresponding to the target preset object is compared with the second feature data. By judging whether the first modal data and the second modal data of the target object are consistent through the similarity of the feature data under the same modality, the accuracy of the result of detecting whether different information representation methods of the same target object are consistent can be improved.

[0022] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0024] Figure 1is one of the flowchart diagrams of an information detection method according to an exemplary embodiment;

[0025] Figure 2 is a second flow chart of an information detection method according to an exemplary embodiment;

[0026] Figure 3 is a third flow chart of an information detection method according to an exemplary embodiment;

[0027] Figure 4 is a fourth flow chart of an information detection method according to an exemplary embodiment;

[0028] Figure 5 is a fifth flow chart of an information detection method according to an exemplary embodiment;

[0029] Figure 6 is a block diagram of an information detection device according to an exemplary embodiment;

[0030] Figure 7 The diagram is a schematic structural diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0031] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0032] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0033] In addition, in the description of the embodiments of the present disclosure, unless otherwise specified, " / " means or, for example, A / B can mean A or B. "And / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present disclosure, "multiple" means two or more than two.

[0034] In the information publishing scenario, assume that the works published by the user account include video files and text files describing the videos. When user account A searches on the platform where the information is published, the search results are displayed based on the search terms entered by user account A. The server usually searches for the text file corresponding to the search term, and then uses the work corresponding to the text file as the search result. If the text file and the video file are inconsistent in the work, the displayed search results are not accurate. Therefore, in order to be able to display the search results accurately, it is also necessary to detect whether the data of different modes (video files and text files) in a work are consistent.

[0035] In the related art, the consistency of text files and video files is detected based on the classification model. However, since the classification model is strongly dependent on the data set, and the data set required for the classification model training method is high-quality confidence samples obtained through manual collection, it is difficult to collect high-quality confidence samples, resulting in the classification model obtained after training may be inaccurate, which in turn leads to inaccurate results in detecting whether different information representation methods of the same object are consistent.

[0036] Based on this, the embodiment of the present disclosure provides an information detection method to solve the above problem. The electronic device that executes the above information detection method can be a personal smart device such as a mobile phone or a tablet computer, or it can also be an electronic device such as a laptop computer, a handheld computer, a desktop computer, an ultra-mobile personal computer (UMPC), a server, or it can also be other electronic devices that can store and process the first modal data and the second modal data of the target object, and there is no limitation on the form of the electronic device.

[0037] The information detection method provided by the embodiment of the present disclosure is described below in conjunction with the accompanying drawings, and the method is exemplarily illustrated by taking the execution subject as an information detection device as an example.

[0038] Figure 1 The following is a flow chart of the information detection method provided by the embodiment of the present disclosure. Figure 1 As shown, the information detection method provided by the embodiment of the present disclosure includes the following steps 101 to 105.

[0039] Step 101: An information detection device obtains first feature data of a target object according to a first single-mode dual-tower model.

[0040] In the disclosed embodiment, the first characteristic data is characteristic data of the first modal data corresponding to the target object; the target object is a multimedia resource object.

[0041] In the embodiments of the present disclosure, the target object may be represented by data of multiple modalities, wherein the first modal data is one of the ways to represent the target object. It should be noted that each source or form of information can be called a modality, such as smell, hearing, vision, touch, etc., or voice, video, text, etc., or radar, infrared, Bluetooth, broadcast, etc., each of which can be called a modality. In the embodiments of the present disclosure, the modality is used to represent the data type.

[0042] Exemplarily, a work is published on an information publishing platform, and the work includes a video file and a text file for describing the video. Then the first modal data can be a video file or a text file.

[0043] Optionally, in an embodiment of the present disclosure, the first modal data is any one of the following: text data, video data, audio data, and image data. The technical solution provided by the present disclosure brings at least the following beneficial effects: by setting the first modal data to be a plurality of data types, the present application can detect information of a plurality of data types, so as to improve the diversity of information detection.

[0044] In the disclosed embodiment, the first single-modal dual-tower type is used to extract feature data of first modal data, where the first modal data refers to data having the same data type.

[0045] It should be noted that the first single-mode dual-tower model refers to using two different sub-models to process the input data at the same time, and then reprocessing the input results of the two sub-models to obtain the final output result. The dual-tower model can avoid the problem of inaccurate feature extraction results caused by a single sub-model.

[0046] Step 102: The information detection device obtains second feature data of the target object according to the second single-mode dual-tower model.

[0047] In the disclosed embodiment, the second feature data is feature data of the second modal data corresponding to the target object.

[0048] Optionally, in an embodiment of the present disclosure, the second modal data is any one of the following: text data, video data, audio data, and image data. The technical solution provided by the present disclosure brings at least the following beneficial effects: by setting the second modal data to be a plurality of data types, the present application can detect information of a plurality of data types, so as to improve the diversity of information detection.

[0049] In the disclosed embodiment, the data types of the first modal data and the second modal data may be the same or different, such as the first modal data is text data and the second modal data is also text data, or the first modal data is text data and the second modal data is video data. It is understandable that even if the data types of the first modal data and the second modal data are the same, the first modal data and the second modal data are different.

[0050] This step is similar to the above step 101 and will not be repeated in the embodiment of the present disclosure.

[0051] Step 103: The information detection device searches for a target preset object corresponding to the target object in the preset object set according to the first feature data.

[0052] In an embodiment of the present disclosure, the similarity between the third feature data and the first feature data of the target preset object is greater than or equal to a first threshold, and the third feature data is the first modal feature data of the first modal data corresponding to the target preset object; the preset object set includes at least one preset object, and the feature data of the first modal data corresponding to the preset object matches the feature data of the second modal data corresponding to the preset object.

[0053] In an embodiment of the present disclosure, a preset object set includes preset objects. The preset objects are similar to target objects and can be represented by data in multiple modalities. Each preset object is represented by first modal data and second modal data, wherein the first modal data corresponding to the preset object matches the second modal data corresponding to the preset object.

[0054] It should be noted that the preset objects in the preset object set are pre-collected through manual review or other review methods, and meet the collection conditions: the first modal data corresponding to the preset object and the second modal data corresponding to the preset object match (are consistent).

[0055] In the disclosed embodiment, to search for a target preset object, it is also necessary to calculate the first modal similarity between the first feature data and the feature data of the first modal data of any preset object in the preset object set according to a preset algorithm. According to the data characteristics of the first modal feature data, the first modal similarity can be semantic similarity, image similarity, volume similarity, audio similarity, etc.

[0056] In the embodiment of the present disclosure, the first similarity of each preset object in the preset object set is compared with a first threshold, and the preset object corresponding to the first similarity greater than or equal to the first threshold is determined as the target preset object.

[0057] Optionally, in an embodiment of the present disclosure, if there is no first similarity greater than or equal to the first threshold in the preset object set, then preset objects corresponding to a preset number (at least 1) of first similarities are determined as target preset objects in order of the first similarities from high to low.

[0058] It should be noted that since the preset objects in the preset object set are limited, all preset objects may be different from the target object and cannot be used to determine whether the first modal data of the target object matches the second modal data corresponding to the target object. Therefore, as a second best option, the preset object closest to the target object is selected as the basis for judgment.

[0059] Step 104: The information detection device obtains a second modal similarity between the target preset object and the target object according to the fourth feature data and the second feature data of the target preset object.

[0060] In the embodiment of the present disclosure, the fourth characteristic data is characteristic data of the second modal data corresponding to the target preset object.

[0061] Optionally, in the embodiment of the present disclosure, whether the target preset object is determined by the first modality similarity being greater than or equal to the first threshold, or by the first modality similarity being arranged in descending order, when the target preset object is greater than 1, Figure 2 As shown, step 104 can be implemented through steps 201 and 202 .

[0062] Step 201: The information detection device obtains the third modal similarity between each target preset object and the target object according to the fourth feature data and the second feature data of each target preset object.

[0063] Step 202: The information detection device determines the average value of the third modality similarity between each target preset object and the target object as the second modality similarity between the target preset object and the target object.

[0064] In the embodiment of the present disclosure, if the number of target preset objects is greater than 1, in order to avoid the influence of a single target preset object on the second modal similarity due to factors such as inapplicability of the adopted algorithm, calculation deviation or inaccurate data itself, the method of calculating the average value can be adopted.

[0065] The technical solution provided by the present disclosure brings at least the following beneficial effects: by calculating the average value of the third modal similarities of multiple target preset objects, the factors of inapplicability of the similarity calculation method, calculation deviation or inaccurate data itself are reduced, thereby improving the accuracy of the obtained second modal similarity.

[0066] In the disclosed embodiment, based on the data characteristics of the second modality feature data, the second modality similarity may be semantic similarity, image similarity, volume similarity, audio similarity, and the like.

[0067] Step 105: When the second modality similarity is greater than a second threshold, the information detection device determines that the first modality data corresponding to the target object matches the second modality data corresponding to the target object.

[0068] In an embodiment of the present disclosure, the second modal similarity is compared with a second threshold value. When the second modal similarity is greater than the second threshold value, it is determined that the first modal data corresponding to the target object matches the second modal data corresponding to the target object. When the second modal similarity is less than or equal to the second threshold value, it is determined that the first modal data corresponding to the target object does not match the second modal data corresponding to the target object.

[0069] In the disclosed embodiment, the target object is set to correspond to the first modal data and the second modal data, with the purpose of determining whether the first modal data corresponding to the target object matches the second modal data corresponding to the target object. Similarly, if the target object also corresponds to the third modal data, then it is also necessary to calculate the third modal similarity between the feature data of the third modal data of the target preset object and the feature data of the third modal data of the target object, and finally, when the second modal similarity is greater than the second threshold value, and the third modal similarity is greater than the third threshold value, it is determined that the first modal data, the second modal data, and the third modal data corresponding to the target object match. It should be noted that if the target object also includes data of other modalities, then similar to the above steps, it is also necessary to calculate the modal similarity corresponding to the other modal data, and the comparison result of the modal similarity and other threshold values.

[0070] Exemplarily, the first modal data corresponding to the target object is a video file (Yao Ming dunks), and the second modal data corresponding to the target object is a text file (Yao Ming teaches children how to dunk). When using the classification model to directly determine, it may be considered that the video file and the text file are consistent. However, using the method of steps 101 to 105 above, it can be determined that the video file and the text file are inconsistent, because the video content is Yao Ming dunks, but the text description is Yao Ming teaches children how to dunk.

[0071] The technical solution provided by the present disclosure brings at least the following beneficial effects: for different modal data corresponding to the same target object, the first feature data and the second feature data are extracted respectively, and then the target preset object corresponding to the third feature data similar to the first feature data is found under the same modality, and then the fourth feature data of the second modal data corresponding to the target preset object is compared with the second feature data, and through the similarity of the feature data under the same modality, it is judged whether the first modal data and the second modal data of the target object are consistent, which can improve the accuracy of the result of detecting whether different information representation methods of the same target object are consistent.

[0072] Optionally, in order to improve the data credibility of the first feature data and the second feature data, as Figure 3 As shown, before step 101 , the information detection method provided by the embodiment of the present disclosure further includes steps 301 and 302 .

[0073] Step 301: The information detection device determines a first single-modal dual-tower model according to a data type of the first modal data corresponding to the target object.

[0074] In the disclosed embodiment, the first single-modal dual-tower model is trained based on first sample data, and the data type of the first sample data is the same as the data type of the first modal data.

[0075] In the embodiment of the present disclosure, the sub-model in the first unimodal dual-tower model can be configured according to the data type. For example, if the data type is video, the sub-model can select a convolutional neural network algorithm. If the data type is text, the sub-model can select a semantic similarity algorithm.

[0076] Step 302: The information detection device determines a second single-modal dual-tower model according to the data type of the second modal data corresponding to the target object.

[0077] In the disclosed embodiment, the second single-modal dual-tower model is trained based on second sample data, and the data type of the second sample data is the same as the data type of the second modal data.

[0078] In the disclosed embodiment, the second unimodal dual-tower model is similar to the first unimodal dual-tower model. The sub-model in the second unimodal dual-tower model can be configured according to the data type. For example, if the data type is video, the sub-model can select a convolutional neural network algorithm. If the data type is text, the sub-model can select a semantic similarity algorithm.

[0079] The technical solution provided by the present disclosure brings at least the following beneficial effects: before obtaining the first feature data and the second feature data, the module used to obtain the feature data is pre-trained to make the obtained first feature data and the second feature data more accurate.

[0080] Optional, such as Figure 4 As shown, before step 101 , the information detection method provided by the embodiment of the present disclosure further includes steps 401 and 402 .

[0081] Step 401: The information detection device extracts feature data of first modal data corresponding to each preset object in the preset object set according to the first single-modal dual-tower model.

[0082] Step 402: The information detection device extracts feature data corresponding to the second modal data corresponding to each preset object in the preset object set according to the second single-modal dual-tower model.

[0083] In the embodiment of the present disclosure, the preset object set is used to store the preset objects. Since the feature data of the first modal data of the preset objects and the feature data corresponding to the second modal data corresponding to the preset objects are often used in the information detection process, after the preset object set is generated, the above data can be extracted and stored to improve the speed of information detection.

[0084] The technical solution provided by the present disclosure brings at least the following beneficial effects: extracting and storing feature data of the first modal data corresponding to each preset object in the preset object set, and feature data of the second modal data corresponding to the preset object, so as to directly use the above data in the subsequent information detection process, thereby improving the speed of information detection.

[0085] Optional, such as Figure 5 As shown, in Figure 1 On the basis of the method shown, after step 105 , the information detection method provided by the embodiment of the present disclosure further includes step 501 .

[0086] Step 501: The information detection device adds a target object to a preset object set and records first feature data and second feature data.

[0087] The technical solution provided by the present disclosure brings at least the following beneficial effects: since the target object that matches the first modal data and the second modal data has been determined and meets the screening conditions of the preset objects in the preset object set, adding the target object to the preset object set can expand the scope of the preset object set and can improve the accuracy of the results of detecting whether different information representation methods of the same target object are consistent.

[0088] The above mainly introduces the scheme of the embodiment of the present disclosure from the perspective of the method. It is understandable that in order to realize the above functions, the information detection device includes at least one of the hardware structure and software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiment disclosed in this article, the present disclosure can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present disclosure.

[0089] The disclosed embodiment can divide the information detection device into functional units according to the above method example. For example, each functional unit can be divided according to each function, or two or more functions can be integrated into one processing unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional units. It should be noted that the division of units in the disclosed embodiment is schematic and is only a logical functional division. There may be other division methods in actual implementation.

[0090] Figure 6 FIG. 1 is a schematic diagram showing the structure of an information detection device according to an exemplary embodiment. Figure 6 As shown, the information detection device provided by the embodiment of the present disclosure includes an acquisition unit 61, a search unit 62 and a determination unit 63;

[0091] The acquisition unit 61 is used to acquire first feature data of a target object according to the first single-mode dual-tower model, where the first feature data is feature data of first modal data corresponding to the target object, and the target object is a multimedia resource object; for example, Figure 1 As shown, the acquisition unit 61 can be used to execute step 101.

[0092] The acquisition unit 61 is further used to acquire second characteristic data of the target object according to the second single-mode dual-tower model, where the second characteristic data is characteristic data of the second modal data corresponding to the target object; for example, Figure 1 As shown, the acquisition unit 61 can be used to execute step 102 .

[0093] The search unit 62 is used to search for a target preset object corresponding to the target object in the preset object set according to the first feature data obtained by the acquisition unit 61, the first modal similarity between the third feature data of the target preset object and the first feature data is greater than or equal to the first threshold, and the third feature data is feature data of the first modal data corresponding to the target preset object; the preset object set includes at least one preset object, and the feature data of the first modal data corresponding to the preset object matches the feature data of the second modal data corresponding to the preset object; for example, Figure 1 As shown, the search unit 62 can be used to execute step 103 .

[0094] The acquisition unit 61 is further configured to obtain the second modal similarity between the target preset object and the target object according to the fourth feature data and the second feature data of the target preset object found by the search unit 62, wherein the fourth feature data is feature data of the second modal data corresponding to the target preset object; for example, Figure 1 As shown, the acquisition unit 61 can be used to execute step 104 .

[0095] The determining unit 63 is configured to determine whether the first modality data corresponding to the target object matches the second modality data corresponding to the target object when the second modality similarity obtained by the obtaining unit 61 is greater than a second threshold. Figure 1 As shown, the determination unit 63 can be used to execute step 105 .

[0096] Optional, such as Figure 6 As shown, when the number of target preset objects is greater than 1, the acquisition unit 61 is specifically used to: obtain the third modal similarity between each target preset object and the target object according to the fourth feature data and the second feature data of each target preset object; and determine the average value of the third modal similarity between each target preset object and the target object as the second modal similarity between the target preset object and the target object. Figure 2 As shown, the acquisition unit 61 can be used to execute step 201 and step 202.

[0097] Optional, such as Figure 6As shown, the above-mentioned information detection device also includes: a determination unit 63, which is also used for the acquisition unit 61 to determine the first single-modal twin-tower model according to the data type of the first modal data corresponding to the target object before acquiring the first feature data of the target object according to the first single-modal twin-tower model, wherein the first single-modal twin-tower model is obtained by training based on the first sample data, and the data type of the first sample data is the same as the data type of the first modal data; and to determine the second single-modal twin-tower model according to the data type of the second modal data corresponding to the target object, wherein the second single-modal twin-tower model is obtained by training based on the second sample data, and the data type of the second sample data is the same as the data type of the second modal data. For example, Figure 3 As shown, the determination unit 63 can be used to execute step 301 and step 302.

[0098] Optional, such as Figure 6 As shown, the above-mentioned information detection device also includes an extraction unit; the processing unit 64 is also used for the search unit 62 to extract and store the feature data of the first modal data corresponding to each preset object in the preset object set according to the first single-modal dual-tower model before searching for the target preset object corresponding to the target object in the preset object set according to the first feature data; and extract and store the feature data corresponding to the second modal data corresponding to the preset object set and each preset object according to the second single-modal dual-tower model. For example, Figure 4 As shown, the processing unit 64 can be used to execute step 401 and step 402.

[0099] Optional, such as Figure 6 As shown, the above-mentioned information detection device also includes an adding unit 65; the adding unit 65 is used to add the target object to the preset object set and record the first feature data and the second feature data after the determining unit 63 determines that the first modal data corresponding to the target object matches the second modal data corresponding to the target object when the second modal similarity is greater than the second threshold. Figure 5 As shown, the adding unit 65 can be used to perform step 501.

[0100] Optionally, in the above-mentioned information detection device, the first modal data is any one of the following: text data, video data, audio data and image data; the second modal data is any one of the following: text data, video data, audio data and image data.

[0101] Regarding the device in the above embodiment, the specific manner in which each unit performs the operation has been described in detail in the embodiment of the method, and will not be elaborated here.

[0102] Figure 7 Schematic diagram of the structure of an electronic device provided by the present disclosure. Figure 7The electronic device may include a processor 71 and a memory 72 for storing executable instructions of the processor 71; wherein the processor 71 is configured to execute the instructions to implement the information detection method in the above embodiment.

[0103] In addition, the electronic device may further include a communication bus 73 and at least one communication interface 74 .

[0104] The processor 71 may be a central processing unit (CPU), a microprocessing unit, an application specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the disclosed solution.

[0105] The communication bus 73 is a signal path for transmitting information between the above components.

[0106] The communication interface 74 uses any transceiver-like device for communicating with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area networks (WLAN), etc.

[0107] The memory 72 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 72 may exist independently and be connected to the processor 71 via a communication bus 74. The memory 72 may also be integrated with the processor 71.

[0108] The memory 72 is used to store instructions for executing the disclosed solution, and the execution is controlled by the processor 71. The processor 71 is used to execute the program or instructions stored in the memory 72, so as to realize the functions in the disclosed method.

[0109] As an example, combining Figure 6 The functions implemented by the acquisition unit 61, the search unit 62 and the determination unit 63 in the information detection device are similar to those Figure 7 The function of processor 71 in is the same.

[0110] In a specific implementation, as an embodiment, the processor 71 may include one or more CPUs, such as Figure 7 CPU0 and CPU1 in.

[0111] In a specific implementation, as an embodiment, the electronic device may include multiple processors 71, each of which may be a single-CPU processor or a multi-CPU processor. The processor 71 herein may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0112] In a specific implementation, as an embodiment, the electronic device may further include an output device 75 and an input device 76. The output device 75 communicates with the processor 71 and may display information in a variety of ways. For example, the output device 75 may be a liquid crystal display (LCD), a light emitting diode (LED) display device, a cathode ray tube (CRT) display device, or a projector. The input device 76 communicates with the processor 71 and may receive user input in a variety of ways. For example, the input device 76 may be a mouse, a keyboard, a touch screen device, or a sensor device.

[0113] Those skilled in the art will understand that Figure 7 The structure shown in the figure does not constitute a limitation on the electronic device, and may include more or less components than shown in the figure, or combine certain components, or adopt a different component arrangement. Figure 7 The electronic device in the system may be a server, a client or other device.

[0114] In addition, the present disclosure also provides a readable storage medium, which stores a program or instruction. When the instructions in the readable storage medium are executed by the processor, the electronic device can perform the information detection method provided in the above embodiment. Optionally, the readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0115] In addition, the present disclosure also provides a computer program product, including a computer program / instructions, which is stored in a non-volatile readable storage medium. When the computer program product is executed by at least one processor, the electronic device executes the information detection method provided in the above embodiment.

[0116] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0117] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An information detection method, characterized in that: include: According to the first single-modal dual-tower model, first feature data of a target object is acquired, where the first feature data is feature data of first modal data corresponding to the target object, and the target object is a multimedia resource object; According to the second single-mode dual-tower model, second feature data of the target object is obtained, where the second feature data is feature data of the second modal data corresponding to the target object; According to the first feature data, searching for a target preset object corresponding to the target object in the preset object set, wherein a first modality similarity between third feature data of the target preset object and the first feature data is greater than or equal to a first threshold, and the third feature data is feature data of first modality data corresponding to the target preset object; the preset object set includes at least one preset object, and feature data of first modality data corresponding to the preset object matches feature data of second modality data corresponding to the preset object; obtaining, according to the fourth feature data and the second feature data of the target preset object, a second modal similarity between the target preset object and the target object, wherein the fourth feature data is feature data of the second modal data corresponding to the target preset object; when the number of the target preset objects is greater than 1, obtaining, according to the fourth feature data and the second feature data of the target preset object, the second modal similarity between the target preset object and the target object, comprises: obtaining, according to the fourth feature data and the second feature data of each of the target preset objects, a third modal similarity between each of the target preset objects and the target object; and determining an average value of the third modal similarities between each of the target preset objects and the target object as the second modal similarity between the target preset object and the target object; When the second modality similarity is greater than a second threshold, it is determined that the first modality data corresponding to the target object matches the second modality data corresponding to the target object.

2. The information detection method according to claim 1, characterized in that: Before acquiring the first characteristic data of the target object according to the first single-mode dual-tower model, the method further includes: Determining the first single-modal dual-tower model according to the data type of the first modal data corresponding to the target object, wherein the first single-modal dual-tower model is obtained by training according to the first sample data, and the data type of the first sample data is the same as the data type of the first modal data; The second single-modal dual-tower model is determined according to the data type of the second modal data corresponding to the target object, wherein the second single-modal dual-tower model is trained based on the second sample data, and the data type of the second sample data is the same as the data type of the second modal data.

3. The information detection method according to claim 2, characterized in that: Before searching the preset object set according to the first feature data for the target preset object corresponding to the target object, the method further includes: Extracting and storing feature data of first modal data corresponding to each preset object in the preset object set according to the first single-modal dual-tower model; According to the second single-modal dual-tower model, feature data corresponding to the second modal data corresponding to each preset object in the preset object set is extracted and stored.

4. The information detection method according to claim 1, characterized in that: After determining that the first modality data corresponding to the target object matches the second modality data corresponding to the target object when the second modality similarity is greater than a second threshold, the method further includes: The target object is added to the preset object set, and the first feature data and the second feature data are recorded.

5. The information detection method according to any one of claims 1 to 4, characterized in that: The first modal data is any one of the following: text data, video data, audio data and image data; the second modal data is any one of the following: text data, video data, audio data and image data.

6. An information detection device, characterized in that: It includes an acquisition unit, a search unit and a determination unit; The acquisition unit is used to acquire first feature data of a target object according to a first single-modal dual-tower model, wherein the first feature data is feature data of first modal data corresponding to the target object, and the target object is a multimedia resource object; The acquisition unit is further used to acquire second feature data of the target object according to the second single-mode dual-tower model, where the second feature data is feature data of the second modal data corresponding to the target object; The searching unit is configured to search, in a preset object set, a target preset object corresponding to the target object according to the first feature data acquired by the acquiring unit, wherein a first modality similarity between third feature data of the target preset object and the first feature data is greater than or equal to a first threshold, and the third feature data is feature data of first modality data corresponding to the target preset object; the preset object set includes at least one preset object, and feature data of first modality data corresponding to the preset object matches feature data of second modality data corresponding to the preset object; The acquisition unit is further used to obtain the second modal similarity between the target preset object and the target object according to the fourth feature data and the second feature data of the target preset object found by the search unit, wherein the fourth feature data is feature data of the second modal data corresponding to the target preset object; when the number of the target preset objects is greater than 1, the acquisition unit is specifically used to: obtain the third modal similarity between each of the target preset objects and the target object according to the fourth feature data and the second feature data of each of the target preset objects; and determine the average value of the third modal similarity between each of the target preset objects and the target object as the second modal similarity between the target preset object and the target object; The determining unit is configured to determine that the first modality data corresponding to the target object matches the second modality data corresponding to the target object when the second modality similarity acquired by the acquiring unit is greater than a second threshold.

7. The information detection device according to claim 6, characterized in that: The device also includes: The determination unit is also used for the acquisition unit to determine the first unimodal twin-tower model according to the data type of the first modal data corresponding to the target object before acquiring the first feature data of the target object according to the first unimodal twin-tower model, wherein the first unimodal twin-tower model is trained based on first sample data, and the data type of the first sample data is the same as the data type of the first modal data; and to determine the second unimodal twin-tower model according to the data type of the second modal data corresponding to the target object, wherein the second unimodal twin-tower model is trained based on second sample data, and the data type of the second sample data is the same as the data type of the second modal data.

8. The information detection device according to claim 7, characterized in that: The device further comprises: a processing unit; The processing unit is also used for the search unit to extract and store feature data of the first modal data corresponding to each preset object in the preset object set according to the first single-modal dual-tower model before searching for the target preset object corresponding to the target object in the preset object set according to the first feature data; and to extract and store feature data corresponding to the second modal data corresponding to each preset object in the preset object set according to the second single-modal dual-tower model.

9. The information detection device according to claim 6, characterized in that: The device further comprises: an adding unit; The adding unit is used for adding the target object to the preset object set and recording the first feature data and the second feature data after the determining unit determines that the first modal data corresponding to the target object matches the second modal data corresponding to the target object when the second modal similarity is greater than a second threshold.

10. The information detection device according to claim 6, characterized in that: The first modal data is any one of the following: text data, video data, audio data and image data; the second modal data is any one of the following: text data, video data, audio data and image data.

11. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the information detection method as described in any one of claims 1-5.

12. A readable storage medium, characterized in that: When the instructions in the readable storage medium are executed by a processor, the information detection method according to any one of claims 1 to 5 can be implemented.

13. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the information detection method according to any one of claims 1 to 5 is implemented.

Images